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WORKSHOP  BACKGROUND 


Ever  since  the  movie  Star  Wars  showed  Luke  Skywalker  and 
R2D2  teaming  up  to  destroy  the  Death  Star,  there  has  been 
considerable  speculation  as  to  how  an  efficient  pilot-robot  team 
could  be  created.  Since  weight  is  a  critical  design  factor  in 
airborne  systems,  the  literal  building  of  a  pilot-robot  team  has 
not  been  undertaken;  rather,  the  emphasis  shifted  to 
incorporating  the  intelligence  of  the  robot.  As  work  in  this 
area  progressed,  such  terms  as  "electronic  crewmember"  and  "black 
box  back  seater"  began  to  enter  the  vocabulary  of  both  the 
crewstation  design  and  computer  software  communities.  While  the 
use  of  these  titles  served  to  stimulate  thinking  in  the  area  of 
human-computer  teamwork,  a  major  program  was  needed  to  start  the 
design  and  implementation  of  concepts  needed  to  build  an 
electronic  crewmember  (EC) ;  in  the  US  this  took  the' form  of  the 
Pilot's  Associate  Program.  The  establishment  of  the  Pilot's 
Associate  Program  in  1985  gave  credence  to  the  idea  that  the 
building  of  the  brain  of  R2D2,  in  some  very  simplified  form, 
might  be  possible. 

In  the  next  two  years,  numerous  discussions  were  held  to 
explore  some  of  cockpit  ramifications  created  by  the  use  of  a 
pilot-EC  team  within  the  aircraft.  These  discussions  occurred 
in  various  technical  meetings  within  the  US  and  the  UK.  In  one 
of  the  meetings  held  in  the  US,  attended  by  representatives  of 


the  Air  Force  of  the  Federal  Republic  of  Germany  as  well  as  US 
and  UK  representatives,  the  idea  of  the  present  workshop  was 
born.  Although  progress  on  the  idea  of  a  workshop  concerning 
human-EC  teamwork  continued,  in  1987  an  event  occurred  which 
demonstrated  the  definite  need  for  the  workshop. 

In  April  of  1987,  USAF  representatives  gave  a  paper  at  a 
meeting  of  the  Royal  Aeronautical  Society  in  London  and  again  at 
a  meeting  of  the  Ergonomics  Society  in  Swansea,  Hales.  The 
subject  of  the  paper  was  "Workload  and  Situation  Awareness  in 
Future  Aircraft",  and  a  section  of  the  paper  discussed  workload 
sharing  between  the  pilot  and  the  EC.  During  both  meetings  the 
same  kinds  of  questions  were  asked:  Is  the  pilot  always  in 
charge?  Can  the  pilot  and  EC  really  be  called  a  team?  Why  do  you 
need  the  pilot  at  all? 

These  thought  provoking  questions  resulted  in  continued 
discussions  with  technical  personnel  in  the  US,  UK  and  FRG,  and 
served  to  provide  a  focus  for  the  workshop.  Through  these 
discussions,  sponsorship  was  obtained  from  organizations  within 
the  three  Air  Forces,  and  as  a  result  the  workshop,  which  the 
German  Air  Force  generously  agreed  to  host,  became  a  reality. 


EXECUTIVE  SUMMARY 


The  meeting  was  divided  into  two  sections:  formal  presentations 
(papers)  and  workshop.  The  papers  covered  a  wide  range  of  topics 
ranging  from  artificial  intelligence  (AI)  implementation  issues, 
through  pilot-electronic  crewmemoer  (EC)  dialogue,  to  the  EC's 
autonomy  and  ouilding  trust  oetween  the  two  crew  members.  A 
summary  of  the  ideas  of  the  French,  German,  British,  and  American 
papers  is  given  below. 

Although  only  one  French  representative  participated,  his  paper 
was  quite  germane  to  the  subject  of  the  meeting.  He  presented  a  very 
interesting  concept  called  tne  "Electronic  Co-Pilot"  which  is  being 
offered  as  an  option  for  the  Rafale  aircraft.  While  not  as  sophisticated 
as  tne  Pilot's  Associate,  it  is  being  targeted  for  a  soon-to-be  operational 
system.  Many  questions  were  asked  about  the  French  approach  towards 
implementing  the  Electronic  Co-pilot  (they  apparently  will  not  use  a 
blackboard  system  currently  favored  by  the  Pilot's  Associate  contractors). 

The  German  speakers  discussed,  among  many  topics,  the  knowledge 
engineering  problem  and  presented  a  means  of  automatic  acquisition  of 
knowledge  through  software  that  monitors  pilot  behavior  and  "learns" 
the  pilot's  intent  by  looking  at  patterns  of  switch  activations. 

The  British  speakers  were  quite  concerned  with  the  ability  to 
program  higher  level  intellectual  functions  within  the  EC;  concepts 
such  as  intuition  and  non-rational  decision  making  were  discussed  at 
great  length. 

The  American  Speakers,  possibly  because  they  had  more  practical 
experience  in  implementing  AI  relative  to  aircraft,  concentrate  on 
lessons  learned.  Levels  of  autonomy  within  the  EC,  and  the  building  of 
five  interdependent  expert  systems  functioning  simultaneously  elicited 
a  great  deal  of  discussion. 

After  the  presentation  of  the  papers,  the  second  half  of  the 
meeting  consisted  of  a  workshop;  its  purpose  was  to  address  AI 
technology  issues  and  cockpit  implications  of  the  technology,  in  a 
number  of  small  discussion  groups.  The  workshop  agenda  was  further 
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subdivided  into  state  of  knowledge,  unresolved  issues,  and  potential 
directions,  as  a  total  of  six  grouos  was  formed.  At  the  end  of  the  workshop 
each  of  the  six  team  chairs  presented  the  results  of  their  del iberations. 

The  conclusions  reached  by  each  of  the  six  teams  are  reported  in  Sections  7. 

Below  are  the  overall  views  of  the  different  technical  disciplines  represented 
at  the  workshop. 

Three  technical  disciplines  were  represented  at  the  conference,  namely 
pilots,  crewstation  designers,  and  artificial  intelligence  experts.  Each 
discipline  had  a  unique  view  of  human-computer  interaction.  The  pilots 
expressed  a  healthy  skepticism  of  the  abilities  of  the  EC  and  were  especially 
concerned  that  their  role  as  aircraft  commanders  be  preserved.  The  crewstation 
designers,  on  the  other  hand,  were  primarily  concerned  with  human-computer  dialogs 
how  can  really  effective  communication  be  buitt  up  between  the  two  members  of 
the  crew?  Finally,  the  artificial  intelligence  experts  were  interested  in  the 
tools  needed  to  make  the  EC  smart  and  discussed  both  the  state  of  the  art 
and  the  difficulties  in  implementing  Al  in  the  airborne  environment. 

The  meeting  identified  many  different  approaches  and  alternative  ways  of 
thinking  about  common  problems.  But  there  was  a  considerable  amount  of 
consensus  about  the  state  of  knowledge  and  about  the  major  unresolved  issues. 

The  main  conclusions  are  summarised  in  Sections  3.  Implementation  of  teaming 
concents  for  human-computer  interaction  raises  imt  rtant  issues  for  all  the 
disciplines  represented  at  the  meeting.  Much  uncertainty  remains  to  be  resolved 
before  a  fully  mature  Human -Electonic  Crew  relationship  can  be  achieved. 

The  meeting  provided  a  timely  and  fruitful  forum  for  exchanging  ideas  and 
for  advancing  inter-disciplinary  and  international  understanding  in  the  area 
of  Human-Electronic  Crew  Teamwork. 
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Introduction 

During  the  last  3even  years  a  number  of  activities  took  place 
within  the  NATO  community  which  all  served  the  same  purpose: 

To  extend  and  improve  our  knowledge  and  rules  data  base  serv¬ 
ing  the  task  of  man-machine  interface  engineering  in  design 
and  development  of  high  performance  air  combat  systems. 

The  knowledge  gap  was  first  formulated  during  the  U.S.  Natio¬ 
nal  Academy  of  Science  study  on  "Automation  in  Combat  Air¬ 
craft  ",  held  in  1981  /I/.  It  gave  the  impetus  for  the  study 
of  the  GCP/WG. 07  on  "Improved  Guidance  and  Control  Automation 
at  the  Man-Machine  Interface"  from  1983  to  1985.  The  result¬ 
ing  advisory  Report  AR-No.  228  was  published  in  December 
1986. 

In  April  1984  the  NATO  Defense  Research  Group,  Panel  VIII 
held  a  Workshop  on  "Application  of  System  Development"  in 
Shrivenham,  England.  The  workshop  concentrated  on  advanced 
crew  station  design,  cockpit  automation  technology,  and  ope¬ 
rator  performance  /2/. 

The  40th  GCP  Symposium  on  "Guidance-Control-Navigation  Auto¬ 
mation  for  Night  All-Weather  Tactical  Operations",  held  in 
Den  Haag  21-24  May  1985  was  another  occasion  where  the 
advances  in  automation  and  Man-Machine-Interface  (MMI)  design 
were  reviewed  /3/. 

During  the  same  time  a  number  of  national  research  projects 
were  initiated  in  the  various  NATO  Nations  dealing  with  the 
above  formulated  objective.  These  projects  included  analyses, 
flight  test  programmes,  and  experimental  aircraft  develop¬ 
ments  to  demonstrate  automation  technologies  and  capabilities 
including  the  advances  in  crew  station  integration. 
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In  this  presentation  we  are  looking  at  the  co-operation  be¬ 
tween  the  human  and  the  electronic  pilot  in  the  light  of  the 
papers  of  the  AGARD  Symposium  on  "The  Man/Machine  Interface 
in  Tactical  Aircraft  Design  and  Combat  Automation”.  This  Sym¬ 
posium  was  held  in  Stuttgart  in  late  September  1987.  It  in¬ 
cluded  contributions  from  the  GCP,  the  FMP  and  the  AMP. 

The  objective  of  the  Symposium  was  automation  at  the  Man-Ma¬ 
chine-Interface  (MMI ) .  You  can  talk  and  may  do  a  lot  about 
cockpit  automation  without  even  touching  the  domain  of  arti¬ 
ficial  intelligence  (AI).  However,  when  complex  functions 
shall  be  automated,  as  mission  planning,  sensor  fusion  and 
correlation,  and  situational  awareness,  you  end  up  in  the 
middle  of  the  wide  field  of  AI  application. 
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2.  The  contributions  of  the  AGARD  Symposium  in  Stuttgart 

During  the  Stuttgart  Symposium  25  %  of  the  papers  presented 
dealt  with  cockpit  automation  and  topics  related  to  AI: 

(1)  Cockpit  automation  requirements  and  the  pilot's  role 

o  "Moding  Strategy  for  Cockpit  Data  Management  in  Mo¬ 
dern  Fighter  Aircraft*  (paper  No.  11)  demonstrated  a 
method  by  which  the  operational  requirements  for 
automation  of  cockpit  functions  can  be  derived.  6 
levels  of  automation  are  defined  (see  Annex) 

o  "Pilots  as  System  Managers  and  Supervisors,  a  risky 
new  Role  according  MMI  Reliability"  (No.  17)  illus¬ 
trated  the  importance  of  the  pilot's  "mental  repre¬ 
sentation"  of  his  tasks,  and  how  it  changes  with  ex¬ 
perience,  and  with  increasing  confidence  in  the  sys¬ 
tems  functions  reliability 

o  "Cockpit  Automation  -  A  Pilot’s  Perspective"  (No.  21) 
discussed  several  considerations  (situation  aware¬ 
ness,  automation  philosophy)  for  developing  a  frame¬ 
work  for  assuring  machine  capabilities  to  complement 
inherent  human  abilities  and  talents  rather  then  to 
replace  them. 

Relation  to  AI  application: 

o  Introduction  of  levels  of  automation  for  cockpit 
functions; 

o  Definition  of  the  new  pilot’s  role,  with  relation  to 
mental  modeling  and  mental  representation  of  tasks, 
considering  Anderson’s  /4/  terminology  concerning  the 
"Declarative-Memory",  the  "Production"-  and  the 
"Working-Memory" ; 
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o  Discussion  of  requirements  for  cockpit  automation  to 
improve  situation  awareness  and  to  solve  the  problem 
that  "Today's  fighter  pilot  is  drawing  in  data  and 
starving  for  information". 

(2)  Expert  systems  for  mission  planning 

o  "Mission  Scenarios,  Planning  and  Requirements" 

(No.  3); 

o  "Expert  System  for  Dow  Level  Tactical  Mission  Prepa¬ 
ration"  (No.  4). 

Relation  to  AI  application: 

The  need  for  AI,  of  Expert  Systems  in  particular  is 
emerging  for  mission  and  attack  profile  planning,  becau¬ 
se  of  the  increasing  number  of  factors  be  taken  into 
account. 

These  are: 

o  Mission  related  data  (air  task,  force  allocation, 
target  intelligence) 

o  Situation  related  data  (intelligence,  navigational 
restrictions,  meteorological  conditions) 

o  Permanent  planning  data  (map,  terrain  digitized  ba¬ 
sis,  navigation  aids,  air  bases,  tactical,  weapon 
etc. ) 
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I  \  (3)  Sensor  Fusion 

i 

|  o  "Multisensor  Target  Reconnaissance"  (No.  16).  In 

these  experiments  a  knowledge  based  fusion  system  is 
developed  combining  radar  (as  primary  sensor)  with  IR 
information  to  improve  target  identification; 

o  "Terrain  Referenced  Avionics  and  Sensor  Fusion  -  The 
Key  to  Mission  Success"  (No.  30).  The  investigation 
applies  EO-sensors  in  combination  with  terrain  refe¬ 
rence  data  base  systems  for  low  level  night  opera¬ 
tions. 

Relation  to  AI  application: 

The  tasks  are  navigation  (low  level  and  night),  recon¬ 
naissance  (target  identification  and  classification), 
and  weapon  delivery  (auto  correlation,  target  prioriti¬ 
zation,  weapon  fusing,  integrated  fire-flight-control). 

This  is  the  area  of  paramount  interest  in  to-day's  R  &  D 
concerning  AI  application. 

Algorithms  for  individual  tasks/functions  are  being  de¬ 
veloped  in  various  countries.  However,  what  is  neeeded 
is  an  overall  approach  to  the  development  of  the  automa¬ 
tion  system  providing  a  truly  integrated  capability  and 
an  AI  system  providing  a  true  pilot  support  for  his  si¬ 
tuation  awareness  and  decision  aiding. 

(4)  "Expert  Man/Machine  Interface  in  Combat  Aircraft  Cock¬ 
pit"  (No.  19).  In  this  paper  a  comprehensive  "man-cen¬ 
tered"  approach  was  presented  to  develop  expert  systems 
for  pilot  support  by 

o  analysing  the  information  status 
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o  monitoring  of  pilot  behaviour  (actions,  judgements) 
in  low  level  flight 

o  decision  support 

o  monitoring  the  flight  and  mission  conduct  against  a 
general  qualitative  model  thereof. 

The  progress  achieved  meanwhile  is  the  objective  of  the 
papers  on  the  "Electronic  Copilot"  presented  by  Avions 
Marcel  Dassault-Breguet  Aviation  in  this  Workshop. 
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Resulting  conclusions  and  recommendations 

In  summarizing  the  results  of  the  Stuttgart  Symposium  con¬ 
cerning  cockpit  automation  and  AI  application  the  following 

conclusions  and  recommendations  are  emphasized: 

(1)  New  technologies  have  their  full  benefit  in  terms  of  sy¬ 
stem  effectiveness  only  when  they  are  applied  embedded 
in  the  operational  context  and  given  situation  awarness. 

(2)  Reliability  is  not  a  matter  of  mathematics  only.  One 
single  failure  can  destroy  the  built  up  confidence  into 
a  system  or  technology. 

(3)  The  pilot  does  not  want  to  put  his  life  into  the  hands 
of  automatic  systems.  They  shall  only  aid,  support  and 
protect  him. 

(4)  Onload  man  from  system  and  flight  management  tasks,  and 
make  him  free  for  the  mission. 

(5)  Sensor  fusion  realisation  is  beginning  to  emerge. 
However,  the  tools  (knowledge  and  rule  based  algorithms) 
are  not  developed  as  yet  for  application  to  system  spe¬ 
cifications  for  the  next  generation  fighter  aircraft. 

Sensor  fusion  investigations  were  presented  which  are 
very  promising.  However  a  systematic  concept  needs  to  be 
developed  taking  into  account  typical  sensor  combina¬ 
tions  applicable  to  specific  tasks,  e.g.  threat  assess¬ 
ment,  target  prioritization,  low  level  navigation. 

(6)  The  impact  of  digital  data  bases  coupled  with  AI  systems 
is  fundamentally  profound.  Expert  System  concepts  are 
being  developed  for  planning  and  diagnostic  tasks,  such 
as  mission  planning  or  systems  health  monitoring. 
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The  availability  of  AI  tools  applicable  to  decision  aid¬ 
ing  in  the  cockpit  in  real  time  operations,  such  as  tar¬ 
get  identification,  prioritization,  and  acquisition  will 
take  several  years  to  fully  mature. 

Development  of  real  time  decision  aiding  concepts  should 
be  accelerated  to  provide  the  necessary  total  combat  si¬ 
tuation  awareness. 

(7)  we  must  keep  in  mind  that  AI  is  not  comparable  with 
human  intelligence: 

The  more  knowledge  a  Human  Expert  has  the  faster  he 
works. 

The  more  knowledge  and  Expert  System  has,  the  slower  it 
works ! 

(3)  We  must  keep  in  mind  that  pilot  acceptance,  system  ef¬ 
fectiveness  and  safety  are  of  paramount  importance  in 
introducing  increasing  levels  of  automation. 
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4.  Cockpit  automation  and  the  domain  of  AI 

It  sounds  very  intellectual  talking  of  "Artificial  Intelli¬ 
gence".  Therefore  some  people  talk  of  "Electronic  Intelligen¬ 
ce"  meaning  just  "Automation".  We  should  be  very  careful  in 
defining  the  domain  of  AI  application  and  in  discriminating 
it  against  plain  automation  of  system  functions.  Using  a  di¬ 
gitized  terrain  data  base  in  combination  with  a  threat  intel¬ 
ligence  data  base  for  low  level  flight  and  navigation  control 
is  -  in  my  understanding  -  plain  automation.  It  becomes  AI 
only  if  it  includes  additional  capabilities  as: 

o  A  knowledge  base  of  sensor  and  threat  data  classification, 
and  of  information  on  e.g.  the  consequences  of  navigation 
aid  restrictions; 

o  A  rule  base  and  inference  capability  for  identifying, 
classifying  and  correlating  sensor,  threat  and  stored 
data. 

For  to  differentiate  between  automation  and  AI  we  can  classi¬ 
fy  the  cockpit  functions  and  tasks  based  on  the  terminology 
introduced  by  Rasmussen  /6/  and  Morgan  /5/. 

Rasmussen  distinguishes  for  human  performance  modelling  be¬ 
tween  skill,  rule,  and  knowledge  based  behaviour. 

Morgan  classifies  the  cockpit  functions  into  operations,  de¬ 
cisions,  and  problem  formulations.  He  allocates  these  func¬ 
tion  types  to  the  function  levels  of  Rasmussen. 

In  MBB  we  defined  "levels  of  automation”,  published  in  /!/ 
and  /8 /,  applicable  to  system  functions  and  cockpit  procedu¬ 
res  automation. 

If  we  combine  these  approaches,  the  domain  of  AI  applications 
to  cockpit  tasks  emerges.  This  is  shown  in  the  figure  below. 
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Function  I  Skill  based  I  Rule  based  I  Knowledge  based 
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Classification  of  cockpit  functions  and  tasks  according  to  the  application  of 
automation  and  Al-techniques 


1 1 


In  this  classif ication  diagram  I  only  included  the  real  time 
pilot  tasks. 

The  real  time  "system  health  monitoring"  tasks  were  not  a  to¬ 
pic  of  the  Stuttgart  Symposium.  The  state  of  the  art  in  this 
domain  has  been  reviewed  in  1986  in  an  Air  Force  Workshop  on 
Artificial  Intelligence  Applications  for  Integrated  Diagnos¬ 
tics  /9/. 


Concluding  remarks 

The  evaluation  of  the  papers  presented  at  the  Stuttgart  Sym¬ 
posium  has  shown  that  the  development  of  AI  tools  applicable 
to  decision  aiding  in  the  cockpit  in  real  time  operations 
"will  take  several  years  to  fully  nature”. 

For  the  next  generation  fighter  aircraft  -  being  specified 
to-day  -  I  can  not  even  see  full  IFFC  capability  to  be  in¬ 
stalled.  And  this  is  automation.  IFFC  as  well  as  the  first 
real  time  AI  tools  could  be  available  possibly  for  the  first 
upgrade  of  the  next  generation  fighter  aircraft,  about  ten 
years  from  now. 

Expert  systems  for  mission  planning  are  being  developed  at 
present.  They  could  be  available  within  a  few  years  for  ap¬ 
plication  in  conjunction  with  mission  data  transfer  systems 
for  to-day's  fighter  aircraft  upgrading. 
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ANNEX 


Automation  Levels  for  the  Man-Machine-Interface  Functions 

1.  MANUAL:  Manual  control  functions  without  automatic  augmenta¬ 
tion  or  support;  man  is  performing  the  activity  using  his 
human  faculties  (e.g.  visual  functions,  mental  activities, 
switching  and  data  input  functions,  verbal  communications). 

2.  MANUAL  AUGMENTED:  This  includes: 

-  manual  control  functions  augmented  by  an  automatic  control 
system  (e.g.  Fly-By-Wire,  Nose  Wheel  Steering); 

-  mental  decision  supported  by  an  automation  system  (e.g. 
step  by  step  check  list  on  a  display). 

3.  MANUAL  AUGMENTED  -  AUTOMATICALLY  LIMITED:  manual  control 
functions  augmented  by  an  automatic  control  system  and  limit¬ 
ed  to  prevent  over-control  and  control  errors.  This  includes: 

-  control  limiting  (e.g.  AOA,  g-level,  attitude  or  velocity 
vector  monitoring); 

-  data  entry  formatting  and  validation  checks. 

4.  AUTOMATIC  -  MANUALLY  LIMITED:  automatic  control  functions  li¬ 
mited,  defined  or  overridden  by  manual  parameters  control 
(e.g.  autopilot  attitude  hold  with  superimposed  pilot  con¬ 
trol)  . 

5.  AUTOMATIC  -  MANUAL  SACNTION:  automatic  control  functions  with 
manual  accept/reject  capability  (e.g.  automatic  targets  prio¬ 
ritization  with  pilot  reject/raodify  function). 

6.  AUTOMATIC:  autonomous  automatic  control  functions  (e.g.  sy¬ 
stems  status  continuous  monitoring  and  alerting). 
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SYSTEMS  ENGINEERING  SUPPORT  FOR  AI 
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SUMMARY 


The  autopoietlc  approach  McNaasa  (1986)  propoaad  for  artificial  intelligence  (AI) 
applicationa  in  advanced  aarospaca  cravstation  dasign  la  not  well-suited  to 
praaanc  design  practicas  and  systems  anginaarlng  aathoda.  Practical 
Implementations  of  advanced  alactronic  crewmember  (EC)  concepts  have  to  bridge 
the  gap  between  computer  sciencas  research  and  large-scale  development  practices 
to  produce  a  viable  crew-systsm  interface.  Theory  and  design  techniques  for 
development  and  test  of  large,  distributed,  concurrent  computer  systems 
envisioned  for  AI  applications  are  still  evolving.  Technology  transfer  must  also 
address  management  Issues.  A  central  systems  engineering  management  Issue  is 
identifying  the  functional  partitioning  of  design  team  activities  necessary  to 
produce  a  humane,  intelligent  system.  A  secondary,  related  problem  Is  verifying 
and  validating  component  elements  to  demonstrate  specification  compliance  and 
performance  adequacy.  Finally,  the  problem  of  Integrated  test  and  evaluation 
presents  many  difficulties  which  are  wall  known  to  the  human  factors  specialist 
but  not  previously  faced  by  systems  engineers.  This  paper  will  address  some 
limitations  of  contemporary  systems  engineering  methods  and  management  techniques 
to  meat  the  challenges  of  the  EC.  Recommended  solutions  which  will  be  proposed 
are  as  follows:  new  systems  engineering  management  concepts  (Incorporating  human 
factors  with  IT&V)  and  support  tools  (integrating  analysis,  testing,  and 
speculation  with  prototyping) . 

1.  INTRODUCTION 

A  pilot's  associate  is  more  than  an  expert.  Humane  intelligent  systems  (McNeese, 
1986)  should  anticipate  the  needs  of  the  pilot.  Interpreting  pilot  actions 
demands  an  updated  modal  of  goals  and  plans,  sat  in  the  context  of  the  pilot's 
Intent  structure  (Smith  and  Broadwall,  1988).  Systems  engineering  practices 
(Booze,  Allen,  and  Hamilton,  1986)  are  not  presently  designed  to  produce  that 
kind  of  product,  and  even  if  the  EC  was  Itself  fully  developed.  Integration  of 
that  technology  Into  weapons  production  is  still  a  problem. 

1.1  Today's  System  Engineering  Problem:  Managing  Complexity 

Aircraft  and  other  systems  have  become  complex  both  in  the  diversity  and  the 
Interactions  that  must  be  managed  among  technical  specialists.  To  manage  the 
synthesis  of  multi-disciplinary  work  efforts,  system  management  methods  already 
exist  for  allocating  duties,  communicating  data,  and  coordinating  effort.  To  be 
effective,  the  system  engineering  management  plan  (SEMP)  must  be  explicit, 
publicly  observable,  and  objectively  measurable.  The  SOiF  conceptually  organizes 
integration  of  technical  teams,  their  assignments,  and  various  work  schedules, 
permitting  a  composite  view  of  the  whole  development  enterprise.  Rouse  and  Cody 
(1988)  nicely  describe  the  shortfalls  of  current  man-machine  Integration  practice 
and  suggest  a  more  user-oriented  approach. 

1.2  Tomorrow’s  System  Engineering  Problem:  Managing  Flexibility 

Tomorrow's  systems  present  new  challenges.  Flexible,  adaptive,  self-organizing 
software  le  a  product  that  is  not  present  in  today's  systems.  The  problem  Is  to 
deliver  a  validated,  tested  design  (assuring  certified  performance  does  not 
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degrade  during  use)  and  requires  a  change  in  the  corporate  culture  of  systems 
engineering  management.  Empirical  and  statistical  testing  methods  that 
incorporate  the  use  of  pilots  will  play  a  larger  role. 

While  appropriate  experimental  tasting  methods  are  typically  well-known  in 
behavioral  and  social-science  research ,  such  methods  are  lass  commonly  used  in 
engineering,  computer  science,  and  contracts  administration.  The  need  to  Include 
humans  in  tests  of  end-item  performance  is  not  a  new  Idea,  but  the  level  and 
amount  of  testing  needed  to  assure  proper  EC  performance  must  increase.  Reasons 
for  this  change  need  to  be  understood  by  managers.  Successful  development, 
delivery,  and  use  of  AI  therefore  require  changes  in  contracts  administration  and 
engineering  management  chat  are  as  revolutionary  as  EC  technology. 

2.  CURRENT  PRACTICES 

New  systems  are  evolutionary  upgrades  of  existing  systems.  New  threat 
capabilities  demand  adaptations  and  enhancements  with  technology  insertion. 
However,  within  the  acquisition  cycle,  there  is  a  well-structured  and  linear 
ordering  of  activities  progressing  from  concept  formulation  through  preplanned 
product  improvements  and  then  subsequent  avionics  modernization  efforts. 
Modernization  programs  occur  because  certain  subsystems  become  obsolete  faster 
than  others.  Subsystem  upgrades  are  more  economical  than  total  system 
replacement. 

Complexity  is  now  managed  by  a  strategy  of  divide  and  conquer.  Functional 
requirements  are  defined  that  assure  meeting  specified  mission  needs,  and 
derivative  functions  are  then  Identified  by  hierarchical  partitioning  into 
progressively  more  detailed  subfunctlona .  The  work  effort  is  itself  broken  down 
In  a  similar  fashion.  The  final  result  is  a  set  of  mutually  exclusive  efforts 
and  a  set  of  discrete  end  items  to  be  produced.  The  work  and  its  products  are 
ralatable,  hierarchical  decompositions  that  split  the  whole  into  smaller 
distinctly  separate,  but  often  interacting,  parts. 

System  testing  is  then  done  at  multiple  levels,  starting  with  Individual  hardware 
(H/W) /software  (S/W)  components,  pair-wise  interactions,  and  then  larger 
assembled  groups.  Each  progression  tends  to  Identify  integration  problems  not 
detected  in  the  simpler  levels  of  testing.  Deficiencies  may  result  from 
Incorrect  or  Incomplete  requirements  specification,  inappropriate  design,  and 
implementation  errors.  Inputs  or  environmental  conditions  for  Integrated  system 
level  testing  in  ground-based  facilities  are  also  Incomplete  and  must  therefore 
be  augmented  by  developmental  flight  tests  and  then  operational  flight  tests,  all 
of  which  are  progressively  better  approximations  to  design  limiting  conditions  or 
some  representation  of  anticipated  combat  conditions. 

During  that  progression,  H/W  and  S/W  performance  are  compared  against  the 
specified  functional  requirements.  Comparably  detailed  evaluations  of 
operator-malntainer  behavior  are  rarely  attempted  in  conjunction  with  integrated 
system  teat  activities.  Costs  and  time  for  such  testing  have  typically  been 
considered  prohibitive. 

There  are  two  questions  asked  in  development  and  operational  tests.  The  first 
asks  whether  the  delivered  system  behaves  as  the  specification  states  it  should. 
The  second  asks  whether  achieved  performance  is  adequate  to  meet  mission  needs. 
The  first  la  a  contractual  issue.  The  second  is  an  operational  issue.  Crew 
opinion  may  be  a  factor  in  answering  the  second  question,  but  is  disallowed  in 
answering  the  first  question.  Crew  performance  and  training  requirements  then 
cope  with  H/W  or  S/W  design  shortfalls,  performance  anomalies,  and  other 
unanticipated  quirks  of  system  behavior  which  are  discovered  after  the  fact,  as 
crews  begin  interacting  with  the  final  products  of  development  and  production] 
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Sometimes  craw*  also  discover  system  capabilities  which  ware  not  intentionally 
part  of  designers '  objectives.  These  can  often  be  exploited  for  tactical 
advantage,  sometimes  compensating  for  other  aspects  of  the  product  which  did  not 
neat  design  expectations. 

Such  progressiva  refinement  during  design,  development,  and  production 
engineering  efforts  results  in  increasing  costs  for  introducing  changes.  Thera 
are  several  reasons  for  these  cost  increases.  First,  design  analyses  need  to  be 
redone  to  assure  proposed  changes  meat  specifications  and  do  not  create  new 
problems.  Second,  design  documentation  has  to  be  changed.  Third,  testing  has  to 
be  teaccompllshed  to  assure  that  interaction  of  the  newly  modified  component  with 
other  system  elements  does  not  induce  seme  unanticipated  and  undesirable  behavior 
elsewhere  in  the  system.  The  smaller  the  span  of  potential  interactive  effects, 
the  more  restricted  and  focused  the  tasting.  Clearly,  this  argues  strongly  for 
a  highly  decoupled  and  modular  system.  That  is  not  the  nature  of  the  EC, 
however,  since  the  EC  will  Interact  with  nearly  everything  that  the  pilot  does. 
Worse  yet,  it  has  to  interact  witl.  pilots  too!  So  tasting  becomes  a  critical 
issue:  How  much  is  enough  and  how  can  it  be  made  more  affordable? 

2.1  Prototyping:  Promise  and  Pitfalls 

Valid  requirements  Identified  at  design  start  minimizes  costa.  If  the  system 
specification  was  accurata  at  every  desired  level  of  detail,  than  testing  would 
largely  verify  design  compliance  rather  than  datacting  defects.  Bacause  pilots' 
behsvlor  cannot  be  perfectly  predicted,  empirical  tasting  is  ueedad.  Rapidly 
raconflgurabla  prototypes  are  tools  to  gst  crew-system  Interface  requirements 
specif isd  early. 

Such  prototyping  must  be  tightly  coupled  to  actual  system  development  since 
details  bacons  reinterpreted,  redefined,  and  then  implemented.  The  prototype 
used  for  human  tasting  must  be  compared  against  both  the  design  specification  and 
actual  syatam  bahavior,  especially  whan  anomalies  appesr  in  prototype  testing. 
Audltabls  documentation  la  uasdad. 

Fault  mods  and  failure  analysis  cannot  ba  accomplished  until  design  details  are 
known,  but  pilot  workload  Is  drivan  by  handling  such  interruptions  under  lass 
than  ideal  conditions.  Prototypes  can  implement  hypothetical  load  conditions 
before  detailed  design  occurs  but  cannot  portray  actual  conditions.  The  catalog 
of  actual  causes  and  affects  of  system  malfunction  (and  thair  impact  on  the  crew 
Interface)  will  evolve  as  operational  and  combat  experience  occur. 

Ground-baaed  prototypes  cannot  replicate  avery  interacting  environmental  factor 
that  drives  and  limits  the  craw’s  combat  activity.  No  single  test  environment 
can  fully  treat  every  aspect  of  crav-systam  interaction.  Combined  testing  is 
neadsd,  and  even  that  will  fall  short  of  perfectly  replicating  actual  combat 
conditlona. 

2.2  An  Augmented  Solution:  An  Integrated • Evaluation  Methodology 

Wallace,  Stockenberg,  and  Charetts  (19S7)  present  a  unified  methodology  for 
system  development ,  emphasizing  ths  need  for  multiple  parspactives  in  performing 
design  analyses.  Evaluation  itself  requires  three  perepectlves :  1)  analysis,  2) 
empirical  casting,  end  3)  speculative  modeling.  In  analysis,  mathematical  models 
can  serve  as  surrogates  for  (end  predictors  of)  csstabls  behavior.  In  empirical 
tasting,  two  objectives  can  ba  pursued:  1)  validation  of  design  analysis,  and  2) 
correction  of  the  underlying  models.  The  second  obj active  lays  a  foundation  for 
tha  third  perapectlve:  casta  based  on  speculative  modeling.  Speculative 
modeling  predicts  bahavior  chat  cannot  ba  validatad.  For  axampla,  this  includes 
effects  of  chemical  warfare  agents  and  supra  lethal  dosaa  of  ionizing  radiation. 
Slnca  speculations  should  be  made  from  a  validatad  modal,  modeling  efforts  should 
closely  parallel  prototyping  efforts. 


3.  PROGNOSIS  FOR  THE  FUTURE 


Rous*  sad  Cody  (1988)  propos*  r •orienting  conceptual  design  to  *  ueer-cencered 
approach  so  user  considerations  will  lead  instead  of  lag  detailed  design. 
Second,  they  propos*  supporting  the  reality  of  detailed  design  end  development. 
A  realistic  description  is  available  in  Boehm  (1988).  His  spiral  model 
Incorporates  stagewlse,  evolutionary,  and  transform  models  of  development  as 
special  esses.  Boehm's  model  recognizes  that  system  specifications  change  as 
design  Insights  occur,  and  encountered  problems  are  resolved.  Boehm  (1988) 
observes:  "each  cycle  Is  completed  by  a  review  involving  the  primary  people  or 
organization  concerned  with  the  product." 

A  user-centered  approach  requires  organizational  changes  that  influence  these 
primary  people  concerned  with  the  EC.  Archer  (1970)  referred  to  these  as 
"arbiters"  of  design,  who  identify  what  factors  are  important  and  determine  what 
weight  those  factors  receive  in  design  trade-off  decisions.  Hardware  engineers 
presently  dominate  that  group.  They  are  a  subculture  distinctly  different  from 
computer  scientists  and  human  factors  specialists.  To  change  the  way  systems  are 
produced,  corporate  culture  must  also  be  changed. 

3.1  The  Characteristics  of  Corporate  Culture 

Sathe  (1983)  defines  culture  as  an  Important  set  of  assumptions  commonly  shared 
by  a  community  but  often  left  unstated.  These  assumptions  vary  in  content  and 
strength.  Prevailing  culture  will  Impact  cooperation,  decision  making,  control, 
coosunlcation ,  commitment,  perceptions,  and  rationalization  of  behavior. 
Scheln's  (1983)  model  of  culture  suggests  that  observed  behavior  is  only  the 
first  of  three  levels.  The  second  level  consists  of  Justifications  that  make 
sans*  of  the  first  level.  The  third  level  pertains  to  the  beliefs  and  values 
that  underlie  those  justifications  at  the  second  level.  Knowing  the  beliefs  and 
values  commonly  shared  within  a  culture  is  the  ksy  to  understanding  behavior  and 
its  Justification.  Any  pressure  that  forces  modifications  only  in  observable 
behavior  will  induce  transient,  not  permanent,  changes. 

To  permanently  change  system  engineering  management,  changes  must  occur  in 
beliefs  and  values,  and  become  stated  Instead  of  assumed  so  they  are  shared 
between  disciplines,  challenged  by  each,  and  altered  as  required.  Technology 
advances  will  help  change  engineering  practices. 

3.2  Social  Dimensions  of  Automation  Impacts  on  Engineering 

Rouse  and  Cody  (1988)  not*  that  the  technology  for  design  la  changing  but  so  must 
the  concepts.  Computer  Aided  Engineering  (CAE),  Design  (CAD),  Manufacturing 
(CAM),  and  Test  (CAT)  are  all  progressing  rapidly.  While  telacomeiunicatlons 
permit  shared  distribution  of  information.  Short,  Williams,  and  Christie  (1976) 
Identify  telecommunications  shortcomings  In  resolving  conflicts  between  people, 
which  will  Inevitably  arise  as  part  of  development  engineering  problem  solving. 
Rous*  and  Cody  (1988)  refer  to  fellowship  as  a  social  aspect  of  relationships 
that  are  an  integral  part  of  interdisciplinary  cooperation  in  design  teams . 
While  technology  will  force  some  changes  in  engineering  culture,  other  changes 
may  be  needed. 

3.3  Suggested  Approach:  Agency  of  Chang* 

Systems  are  becoming-software  intensive  (Glassman,  1982  and  Grove,  1982).  Many 
require  Independent  Verification  and  Validation  (IV&V)  efforts  (Southworth  and 
Sapp,  1984).  Doing  IV&V  requires  analysis,  modeling  and  simulation,  and 
prototyping.  On*  approach  to  improving  EC  testing  would  be  to  include  human 
factor*  within  the  IV&V  effort.  This  also  permits  an  early  start  on  both 
instructional  system  and  training  device  development,  other  typlcslly  neglected 
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aspects  of  Integrated  logistics  support.  Three  benefits  could  be  realized. 
First,  continuing  hwan-user  testing  could  parallel  development  and  be  associated 
with  required  software  integration  testa.  This  permits  more  extensive  yet  non- 
disruptive  testing.  Second,  multiple  uses  can  be  made  of  the  human  factors  data 
from  such  tests:  (a)  model  parameterization/validation,  (b)  human  engineering 
evaluation,  and  (c)  training  curriculum  validation.  Third,  culture  changes  could 
be  induced  through  existing  organizational  mechanisms  instead  of  adding  a  wholly 
new  structure  just  to  accomplish  tha  level  of  testing  needed  for  producing  a 
useful  EC.  The  human  engineer  must  then  focus  on  becoming  the  change  agent, 
calling  attantion  to  unstatad  systems  engineering  and  management  assumptions 
about  presumably  sharad  beliefs  and  values  that  need  to  be  reviewed  more 
carefully. 


RE7EREHCES 


Archer,  L.  Bruce,  Technological  Innovation:  A  Methodology.  Design  and  Research 
Unit,  Royal  College  of  Art,  London,  1970. 

Boehm,  Barry  W.  "A  Spiral  Modal  of  Software  Development  and  Enhancement,"  IEEE 
Computer  Magazine,  May,  1988,  pp.  61-72. 

Booze,  Allen,  and  Hamilton,  Systems  Engineering  Management  Guide,  2nd  Edition, 
Defense  Systems  Management  Collage,  Fort  Belvoir,  7A,  December,  1986. 

Glassman,  Steven,  Comparative  Studies  In  Software  Acquisition.  Lexington  Books, 
Lexington,  MA,  1982. 

Grove,  H.  Mark,  "DoD  Policy  for  Acquisition  of  Embedded  Computer  Resources," 
Concepts,  The  Journal  of  Defense  Systems  Acquisition  Management,  Vol.  5,  No.  4, 
Aug.  1982,  pp.  9-36. 

McNeese,  Michael  D,,  "Humana  Intelligence:  A  Human  Factor  Perspective  for 
Developing  Intelligent  Cockpits",  IEEE  AES  Magazine.  September  1986,  pp  6-16. 

Rouse,  William  B.  and  William  J.  Cody,  "On  the  Design  of  Man-Machine  Systems: 
Principles,  Practices,  and  Prospects",  Autonatica.  Vol.  24,  No.  2,  pp.  227-238. 

Satha,  Vijay,  Culture  and  Related  Corporate  Realities:  Text,  Cases,  and  Readings 
on  Organizational  Entry,  Establishment,  and  Change,  Richard  D.  Irwin,  Inc., 
Homewood,  IL,  1983. 

S chain,  Edgar  H.,  "Organizational  Culture:  A  Dynamic  Model,”  Working  Paper 
N«ber  1412-83,  Massachusetts  Institute  of  Technology,  Cambridge,  MA,  February, 
1983. 

Short,  John,  Ederyn  Williams,  and  Bruce  Christie,  The  Social  Psychology  of 
Talacosainnlcationa ,  John  Wileys  &  Sons,  NT,  1976. 

Smith,  David  and  Martin  Broadwell,  "Plan  Coordination  In  Support  of  Expert 
Systems  Integration",  DARPRA  AFWAL  Pilot's  Associate  Related  Papers,  BDM 
Corporation,  McLean  VA,  May  1988,  pp  35-40. 

Southworth ,  D.  and  John  W.  Sapp,  "Orlando  I:  Final  Report,  Panel  B,  Independent 
Verification  and  Validation,"  Joint  Logistic  Commanders'  Workshop  on  Post 
Development  Software  Support  for  Mission  Critical  Computer  Software.  Vol.  II, 
June  1984. 

Wallace,  Robert  H.,  John  E.  Stockenberg,  and  Robert  N.  Charette,  A  Unified 
Methodology  for  System  Development,  McGraw-Hill,  NT,  1987. 


20 


TACTICAL  DECISION  AIDS 

BY 


AN  AI  APPROACH 


C  R  Ovenden 

Research  4  Product  Technology  Department 
Smiths  Industries  Aerospace  4  Defence  Systems  Ltd 
Cheltenham  Glos 
UK 


SUMMARY 

As  the  environment  in  which  military  aircraft  operate  becomes  increasingly  hostile 
and  the  advances  in  avionics  systems  provide  the  crew  with  ever  more  data,  mission 
effectiveness  will  be  put  at  risk  by  the  increase  in  crew  workload.  The  tactical 
decision  aid  (TDA)  is  a  system  designed  to  alleviate  this  workload  problem  by  using 
the  incoming  data  to  supply  the  crew  with  high-level  information,  upon  which,  tactical 
decisions  can  be  made.  This  paper  outlines  the  primary  process  of  the  TDA,  namely 
sensor  fusion  and  mission  planning,  which  have  been  approached  from  an  AI  perspective. 
The  test-bed  for  this  work  has  been  a  ground  attack  mission  and  in  particular  the 
hostile  ingress  and  egress  phases  of  the  mission. 


INTRODUCTION 

In  the  foreseeable  future  the  military  aircraft  will  be  operating  in  an  environment 
in  which  the  numbers  and  sophistication  of  threats  deployed  against  it  will  ensure 
that  the  aircrew  has  a  high  workload.  This  will  be  compounded  by  the  move  toward 
single  operator  aircraft  and  the  increased  ability  of  avionic  systems  to  supply  the 
operator  with  data.  These  factors  could  combine  to  increase  the  operator  workload 

to  the  point  where  the  survival  of  the  aircraft  and  the  ability  to  achieve  the  mission 
are  put  in  jeopardy.  The  sophistication  of  proposed  future  avionic  systems  will 
ensure  that  the  aircrew  will  have  as  much  data  as  possible  upon  which  to  make  any 
tactical  or  strategic  decisions,  but  the  limiting  factor  on  arriving  at  the  correct 

decision  will  probably  be  the  speed  at  which  such  data  can  be  assimilated  and  understood. 

The  purpose  of  a  tactical  decision  aid  (TDA)  is  to  reduce  the  crew  workload  by  converting 
the  incoming  data  into  high-level  information  which  can  form  the  basis  for  a  decision 
making  process.  This  decision  making  process  can  then  be  performed  either  by  the 
crew  or  by  the  TDA,  although  in  the  latter  case,  the  reasoning  processes  must  be 
understandable  to  the  crew,  in  order  that  the  basis  for  decisions  can  be  checked 
and  confidence  in  the  system's  abilities  acquired.  Such  issues  abutt  the  MMI  aspects 
of  such  a  system;  these  aspects  have  not  been  addressed  within  the  current  project. 

This  paper  is  an  overview  of  the  current  state  of  the  TDA  project  at  Smiths  Industries 
Aerospace  4  Defence  Systems  Ltd.  Effort  has  been  concentrated  upon  those  areas  of 

the  TDA  which  are  the  most  novel  in  terms  of  current  avionic  computing.  These  concern 
the  fusing  of  data  to  produce  the  high-level  information  (sensor  fusion)  and  the 
use  of  that  information  by  the  TDA  in  the  decision  making  process  (planning). 

The  next  section  describes  the  scenario  in  which  the  TDA  has  been  tested  and  the 
test-bed  simulation  which  has  been  developed.  Following  that,  the  sensor-fusion 
techniques  which  have  been  investigated  are  discussed  and  the  subsequent  section 
covers  the  planning  functions. 


The  Scenario 

Investigation  of  the  TDA  has  been  limited  to  the  problems  faced  in  a  ground  attack 
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mission  and  In  particular  that  part  of  the  mission  concerned  with  a  low-level  Ingress 
to  the  target  area.  Such  an  ingress  involves  a  flight  of  around  8  km  in  a  time  of 
5  minutes  flying  at  a  height  of  around  100  feet.  In  such  circumstances  the  crew 
face  a  high  workload  not  only  from  attempts  to  counter  threats,  but  also  from  the 
need  to  maintain  altitude  and  avoid  terrain.  Under  these  circumstances  the  crew 
may  not  be  performing  to  the  best  of  their  capabilities  or  those  of  the  aircraft. 

The  principal  threat  in  such  a  mission  comes  from  surface-to-air  missile  (SAM)  sites, 
associated  both  with  airspace  denial  and  point  defence,  and  anti-aircraft  artillery. 
Such  air  defence  units  can  be  highly  mobile  in  their  attempts  to  cover  potential 
targets  such  as  troop  concentrations.  Consequently  the  location  of  such  threats 
will  be  known  to  a  very  limited  degree  of  accuracy,  which  in  turn,  implies  that  the 
value  of  the  prior  intelligence  is  limited.  Although  the  aircraft  carries  counter¬ 
measures  to  decoy  incoming  missiles,  the  increased  sophistication  of  the  threat  implies 
that  the  efficay  of  these  countermeasures  will  be  greatly  reduced,  leading  to  a  lower 
mission  success  rate  and  a  higher  mission  cost. 

The  test-bed  for  the  TDA  is  simulated  using  object-oriented  programni ng  techniques. 
This  enables  a  SAM  site  to  be  modelled  as  an  entity  which  performs  its  own  tasks, 
seeking  and  acquiring  the  aircraft  as  a  target.  Other  objects  such  as  surveillance 
radars,  troops  and  vehicles  are  also  simulated.  Some  of  the  SAM  sites  communicate 
with  each  other  and  with  surveillance  radar  enabling,  a  means  of  representing  a  co¬ 
ordinated  air  defence  system.  All  SAM  sites  have  a  limited  number  of  missiles  and 
take  time  to  re-load.  The  detail  in  the  simulation  means  that  the  air  defence  response 
to  the  aircraft  is  made  unpredictable  by  the  number  of  decisions  being  made.  This 
complexity  helps  test  the  various  responses  of  the  TDA  in  a  relatively  unbiased  manner, 
which  may  not  be  achieved  by  a  coarser  simulation  of  the  world. 

The  simulation  also  models  the  sensors  and  countermeasures  of  the  aircraft,  and  their 
interaction  with  the  radars  of  the  threat  world,  together  with  the  aircraft's  weapons. 


TACTICAL  DECISION  AID 

The  TDA  has  the  task  of  planning  the  hostile  ingress  phase  of  the  mission  subject 
to  the  constraints  imposed  by  the  aircraft  embodied  in  limitations  upon  the  use  of 
fuel,  countermeasures  and  weapons.  In  performing  this  task  the  TDA  has,  initially, 
an  intelligence  database  comprised  of  the  location  and  type  of  known  threats.  Although 
this  is  assumed  to  be  in  error  it  is  used  as  a  basis  for  the  initial  plan  in  the 

absence  of  better  information.  Having  made  a  plan,  including  a  flight  path,  the 
aircraft  flies  the  given  course.  As  it  does  so  the  sensor,  the  are  usually  in  stealth 
mode  until  the  final  target  attack,  pick-up  emissions  from  the  various  ground-based 
radars  searching  for  incoming  aircraft.  The  TDA  pools  and  interprets  this  data  in 

an  attempt  to  update  the  perceived  situation.  Based  upon  the  changes  in  the  perceived 

situation,  the  TDA  re-plans  or  repairs  the  current  plan  to  ensure  that  the  aircraft 

will  achieve  the  mission  goals,  while  keeping  within  the  constraints  imposed. 

Thus,  in  essence,  the  tasks  of  the  TDA  can  be  defined  as  asynchronous  sensor  fusion 
and  planning.  The  underlying  philosophy  of  the  system  is  that  it  should  be  capable 
of  being  interrogated  about  its  decision  making  processes  and  that  the  answers  given 
should  be  understandable  in  the  operator's  terms.  It  is  not  intended  that  this  inter¬ 
action  should  take  place  during  the  course  of  a  mission,  but  rather  during  training 
and  simulation  sessions,  in  order  to  develop  confidence  in  the  system.  This  has 
an  important  impact  upon  the  approach  taken  to  developing  the  system.  In  particular 
this  philosophy  is  common  and  desirable  in  the  field  of  AI  and  this  led  to  the  adoption 
of  these  techniques  above  any  more  numerical  approaches. 


Sensor  Fusion 

The  data  to  be  fused  in  this  case  is  derived  from  the  data  sources  on  the  aircraft. 
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These  are  the  sensors:  the  radar  warning  receiver  (RWR),  the  radar  (active  and  passive 
modes),  the  forward  looking  infra-red  receiver  (FLIR)  and  the  missile  approach  warning 
receiver  (MAWR) .  These  are  backed  up  by  information  sources  such  as  the  initial 
intelligence  data  and  any  incoming  communications.  Although  the  data  is  accumulated 
over  time,  the  TDA  must  always  be  prepared  to  make  an  identification  even  when  the 
quantity  of  evidence  is  minimal.  The  manner  in  which  the  identifications  are  updated 
and  the  accuracy  required,  place  restrictions  upon  the  nature  of  the  inferencing 
process. 

An  important  aspect  of  data  fusion  is  determing  which  pieces  of  data  to  fuse  and 
which  to  keep  separate  because  they  refer  to  something  different.  This  task  becomes 
more  complex  when  the  purpose  of  fusing  the  data  is  to  determine  what  is  being  observed. 
The  method  used  to  handle  this  difficulty  is  to  assume  that  all  reports  from  the 
same  location  refer  to  the  same  source.  This  assumption  is  later  tested  by  the  infer¬ 
encing  method  used  to  identify  the  source. 

All  of  the  sensors  can  produce  spurious  reports  or  noise  and  all  of  the  sensors  produce 
varying  degress  of  error  in  positioning  the  source  of  the  report.  Consequently, 
the  first  task  of  the  sensor  fusion  process,  when  a  sensor  report  is  received,  is 
to  discard  it  if  it  does  not  correspond  sufficiently  closely  with  any  known  types 
of  source.  Clearly,  some  noise  will  remain  in  the  system  and  the  inferencing  mechanism 
used  to  assign  an  identification  to  the  source  will  need  to  be  able  to  handle  this. 
The  next  task  is  to  determine  with  which  previously  identified  source  the  current 
report  is  co-located  within  the  error  bounds  of  the  sensor.  If  it  cannot  be  co-located 
with  what  is  already  detected  then  a  new  source  is  deemed  to  have  been  found. 

Having  determined  that  the  current  sensor  report  arises  from  the  same  location  as 
a  previously  detected  source,  does  not  necessairly  mean  that  it  has  come  from  the 
same  source,  i.e.,  more  than  one  emitting  entity  may  be  located  in  an  area  bounded 
by  the  sensor  errors.  The  inferencing  technique  used  must  be  able  to  handle  this 
conflicting  evidence.  With  each  source,  a  set  of  hypotheses  as  to  its  identity, 
is  stored.  More  than  one  set  may  be  stored  if  it  is  recognized  that  more  than  one 
entity  is  located  at  that  position.  The  inferencing  technique  updates  the  likelihood 
value  of  each  item  in  these  sets  of  hypotheses  and,  whichever  has  the  highest  such 
value,  is  deemed  to  be  the  best  identification  at  any  time.  In  this  manner,  the 
sensor  fusion  process  always  has  an  identification  at  any  time.  In  this  manner, 

the  sensor  fusion  process  always  has  an  identification  of  a  source,  however  small 

the  amount  of  data  received.  Clearly,  if  a  more  accurate  sensor  picks  up  the  source, 
the  position  of  it  can  be  given  with  greater  accuracy. 

A  number  of  inferencing  techniques  have  been  investigated  and  the  one  which  has  been 
able  to  satisfy  the  above  requirements  has  been  the  Dempster-Shafer  evidential  reasoning 
system  (1).  When  the  first  sensor  report  is  received  from  a  location,  a  list  of 
possible  sources  is  created,  each  with  a  likelihood  generated  from  an  evidential 
interval.  Subsequent  sensor  reports  are  used  to  update  this  set  of  hypotheses. 

The  Dempster-Shafer  approach  can  determine  a  measure  of  conflict  in  the  evidence 
being  received,  enabling  multi-source  identification  to  take  place.  On  the  other 

hand,  the  processing  time  and  storage  needs  can  create  difficulties,  which  have  been 

overcome  by  the  use  of  a  controlling  rule-base. 


Planning 

The  planning  system  has  been  designed  with  a  number  of  requirements  in  mind.  Principal 
among  these,  are  the  potential  need  for  understanding,  the  decision  making  processes 
and  the  need  for  apian  to  be  available  at  all  times,  even  when  processing  is  curtailed, 
due  to  lack  of  available  time.  The  former  of  these,  has  influenced  the  knowledge 
representation  within  the  system  and  both  have  influenced  the  plan  production  process. 
A  further  influence  has  been  the  decision  not  to  attempt  to  create  an  optimum  plan 
(in  the  sense  of  using  minimum  resources  and  minimum  threat  exposure),  but  to  produce 
an  acceptable  plan  which,  while  minimising  the  threat  exposure,  remains  within  an 
allowed  use  of  resources.  It  is  envisaged  that  from  such  a  plan  the  optimum  solution 


can  be  developed  by  using  numerical  techniques  in  a  manner  focussed  by  the  planning 
system. 

The  knowledge  within  the  planning  system  is  divided  into  three  types,  depending  upon 
the  time-scale  of  change  In  that  knowledge.  The  first  type  Is  static  threeghowt 

the  duration  of  a  mission.  This  includes  such  things  as  the  terrain  and  the  capabilities 
of  designated  missile  types.  The  second  type  is  that  concerning  the  perceived  world 
view  derived  from  the  data  fusion  process.  These  are  the  objects  in  the  world. 
The  third  type  is  aircraft  and  mission  specific,  and  includes  the  plan  structure 

based  upon  the  position  and  current  expendables  status  of  the  aircraft. 

The  external  world  is  represented  as  a  set  of  objects,  each  with  a  position  type, 

and  status,  in  relation  to  the  aircraft  and  the  mission.  This  enables  the  plan  to 

be  represented  as  a  time-ordered  set  of  scripts  which  are  also  a  series  of  instructions 
such  as  “fly  left  of  object-1",  "suppress  object-2",  “attack  target-object  while 

suppressing  object-3",  etc.  Representing  the  world  in  this  manner  has  meant  that 
the  plan  can  be  described  in  an  understandable  form  and,  since  the  plan  scripts  contain 
all  the  information  used  to  derive  the  instruction,  the  plan  can  be  interrogated 
as  to  the  reasons  for  arriving  at  a  particular  decision. 

The  planner  has  the  task  of  setting  the  values  of  these  instructions  to  achieve  an 
acceptable  plan  using  the  knowledge  at  its  disposal  including  the  knowledge  of  other 
parts  of  the  plan.  This  it  does  by  firing  a  set  of  production  rules  on  the  plan, 

as  if  it  were  a  knowledge  base.  When  no  more  rules  can  be  fired,  the  plan  is  as 

good  as  it  can  be  made.  Using  this  rule-set,  the  problems  of  plan  monitor  and  repair 
are  considerably  reduced.  The  rule-set  is  ordered  in  such  a  manner  that  it  will 
attempt  to  improve  whatever  plan  it  is  given.  Thus,  if  circumstances  change,  the 

rule-set  will  try  to  improve  the  plan  within  these  new  circumstances  without  the 

need  to  check  whether  this  change  affects  the  plan. 

The  other  advantages  gained  from  the  use  of  a  rule-set  in  this  manner,  are  that  whenever 
the  computation  is  curtailed,  there  will  always  be  a  "best-so-far"  plan  available. 


Conclusions 

The  tasks  which  systems  such  as  the  TDA  are  expected  to  perform  are  currently  performed 
by  the  operator  and  the  intention  of  these  systems  is  to  relieve  the  operator  workload 
by  having  the  system  take  over  the  performance  of  these  tasks.  However,  given  the 

nature  of  the  environment  in  which  these  tasks  are  undertaken  and  the  consequences 

of  an  error,  considerable  effort  needs  to  be  expended  on  inducing  confidence  in  the 
system.  This  implies  that  the  output  of  such  systems  is  open  to  scrutiny  by  the 

operators  and  that  the  systems  are  able  to  answer  questions  in  a  manner  which  is 

comfortable  to  the  operator.  In  the  field  of  artificial  intelligence  such  constraints 
are  common  and  deemed  desirable  in  any  system.  It  therefore  seems  reasonable  to 
apply  the  techniques  and  discipline  of  this  field  to  the  problems  associated  with 
developing  these  systems. 

The  development  of  the  TDA  has  followed  this  policy  and  the  results  have  been 
encouraging.  The  data  from  the  sensors  is  fused  to  form  a  threat  scene  which  is 
then  used  by  the  planning  functions.  The  system  produces  acceptable  mission  plans 
which  take  account  of  the  use  of  expendable  resources  while  minimising  the  threat 
to  the  aircraft.  Throughout  these  processes  the  system  is  open  to  interrogation 
and  the  decision  paths  can  be  explained  in  a  manner  understandable  to  the  operator. 
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Adaptive  User  Interfaces  in  Man  Machine  Systems 


K.  Friedrich.  Kraiss* 


1.  Introduction 

Increasing  automation  in  vehicle  and  process  consol  has  the  consequence  that  operators  are  more 
and  more  restricted  to  supervisory  tasks.  This  is  a  possibly  dangerous  development  because  the 
operators  are  taken  out  of  the  loop.  Consequently  they  may  loose  the  skill  to  judge  complex 
situations  and  to  react  in  a  competent  way.  In  Older  to  counteract  this  effect  two  approaches  can 
be  considered.  Firstly  the  operator  can  be  supported  in  performing  cognitive  tasks  by  providing 
artificial  intelligence  functions  to  him  which  facilitate  decision  making  and  problem  solving. 
Secondly  the  operator's  task  may  be  simplified  by  adaptively  matching  user  interface  functions 
as  well  as  system  functions  to  environmental  conditions,  situations,  tasks  or  user  characteristics. 
This  paper  addresses  the  latter  approach  by  reviewing  machine  learning  algorithms  and  their 
applicability  to  adaptive  user  interfaces 


2.  Adaptive  man  machine  systems 

The  architecture  of  a  typical  conventional  supervisory  control  system  is  depicted  in  figure  1.  As 
can  be  seen,  there  are  two  computers  between  the  operator  and  a  specific  task.  One  of  these 
interacts  mainly  with  the  task,  while  the  other  interacts  with  the  human.  This  concept  enables 
considerable  flexibility  in  task  allocation  between  the  human  and  the  system  as  well  as  with 
respect  to  the  interface  layout  However  current  supervisory  control  systems  are  not  prepared  to 
continuously  adapt  to  user  requirements  and  performance.  They  are  rigid  and  static,  Le.,  they  do 
not  adapt  dynamically  to  variations  in  drill,  motivation,  decision  strategy,  risk  taking  behaviour 
or  cognitive  state  of  the  user.  Individual  preferences  of  display  formats  and  contents  are  not 
supported.  A  different  system  architecture  that  traces  operator  actions  and  evaluates  them  on-line 
will  therefore  be  required  in  order  to  achieve  adaptivity  in  man  machine  systems. 

As  already  mentioned  above,  adaptivity  may  be  desireabie  with  respect  to  environmental 
conditions,  situations,  tasks  and  user  characteristics.  While  the  first  three  items  of  this  list  can 
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be  technically  identified  •  given  the  necessary  sensors  are  available  -  the  identification  to  user 
characteristics  poses  a  major  problem.  A  prerequisite  for  the  dynamic  adaptation  of  user 
interfaces  to  individual  needs  is  that  behavioural  characteristics  be  known  to  the  computer.  A 
cooperative  system  architecture  designed  to  provide  this  functionality  is  presented  in  figure  2.  As 
may  be  seen  by  comparison  with  figure  1  the  human  interactive  computer  has  now  been  specified 
in  mote  detail  and  a  set  of  three  data  bases  have  been  added  to  the  system. 

The  world  state  data  base  contains  environmental  conditions  and  operational  phases.  Knowledge 
about  the  system  is  collected  in  a  separate  data  base  which  contains  system  specific  sripts  and 
procedures.  In  addition  there  is  a  data  base  for  possible  operator  goals,  plans  and  scripts.  These 
data  bases  are  essential  for  the  functioning  of  the  human  interactive  computer  which  contains 
five  components:  Operator  model,  interface  management,  operator  error  handling,  adaptive 
operator  aiding  and  system  error  handling. 

Input  for  the  operator  model  is  the  actual  human  performance  in  a  task  as  identified  by  recording 
the  information  provided  to  an  operator  together  with  fire  actions  he  is  deriving  from  them.  Such 
records  allow  interpretation  and  prediction  of  operator  actions  and  discrimination  of  expected 
(explained)  from  unexplained  actions  (errors  or  innovations).  Resource  utilization  is  derived 
from  current  and  projected  on-line  workload  analysis  which  is  needed  to  estimate  of  operator 
performance  in  current  and  potential  future  tasks.  Errors  of  omission  or  commission  are 
identified  by  comparison  with  active  (legal)  scripts  and  goals.  Subsequently  a  suitable 
remediation  level  for  an  error  (monitoring  and  exploration,  error  feedback  (alert,  warning),  active 
prevention  or  automatic  initiation  of  compensatory  actions  )  is  selected  by  the  operator  error 
handling  module.  System  error  handling  is  supported  by  the  provision  of  interactive  diagnostic 
expert  systems.  Adaptive  operator  aiding  and  interface  management  (see  fig.2)  are  the  modules 
which  are  most  interesting  in  die  context  of  this  paper. 

Adaptive  operator  aiding  includes: 

-  Variable  task  assignment  to  man  and  computer, 

-  Task  transformations  (predictions,  display  modality  etc.), 

-  Adaptation  of  dialog  styles  to  the  drills  and  preferences  of  individual  operators, 

•  Consistency  check  of  operator  actions  and  decisions, 

-  Advice  giving  and  decision  support  functions. 

interface  management  includes: 

-  Information  filtering, 

•  Selection  of  sensory  modality , 

-  Display  formatting, 

-  Adaptive  control  of  display  and  message  (alarm)  sequencing. 


Fig  1.  Typical  Supervisory  System  Architecture  (Sheridan  &  Hennessee  1984) 


Fig  2:  Cooperative  Man-Machine-System  Architecture  ( Kraiss  1978;  Rouse  et  al.1987  ) 
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The  main  difficulty  during  the  implementation  of  the  adaptive  functions  mentioned  above  is  the 
establishment  of  a  suitable  user  model  within  a  computer.  One  approach  to  solve  this  problem  is 
the  application  of  trainable  observers.  As  depicted  in  figure  3  operator  behaviour  can  be  observed 
on-line  and  -  after  enough  training  -  be  duplicated  by  such  a  device.  Subsequently  the  trained 
observer  may  then  be  used  as  operator  model  (Freedy  et  al.  1985 ). 


Fig.  3:  Concept  for  an  adaptive  user  interface  based  on  a  trainable  observer 


3.  Connectionist  learning  mechanisms 


Various  technical  solutions  exist  for  the  implementation  of  trainable  observers.  We  restrict  the 
discussion  here  to  a  short  introduction  to  adaptive  filters  and  neural  networks  because  these  are 
used  in  the  case  studies  described  later.  As  the  main  subject  of  this  paper  is  not  the  theory  of 
connectionist  learning  mechanisms,  the  reader  is  refered  to  the  given  references  for  more  Hwaiu 


Figure  4  shows  a  neural  network  in  very  general  form.  In  a  static  net  (no  state  units)  the  flow  of 
information  is  from  plan  units  over  hidden  units  to  the  output  units.  By  special  training 
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procedures  (e.g.  backward  error  propagation)  the  weights  of  the  links  between  "neurons"  are 
adjusted  such  that  the  net  responds  to  each  plan  vector  with  the  desired  output  vector 
(Rumelhardx  et  aL  1986).  Within  certain  limits  neural  nets  are  able  to  generalize,  i.e.,  to  react  in  a 
meaningful  manner  to  plan  vectors,  which  have  not  been  trained  before.  Recurrent  nets,  i.e. 
those  which  include  state  units  and  feedback  connections,  can  even  be  trained  to  produce 
sequences  of  actions  at  the  output  (Jordan  1986). 

Adaptive  filters  (Widrow  &  Stearns  1985),  also  called  linear  machines  (Nilsson  1965),  are 
closely  related  to  the  neural  nets  mentioned  above.  The  main  difference  is  that  in  these  devices 
there  are  no  hidden  layers.  The  classifier  depicted  in  Figure  5  consists  ,  e.g.,  of  three  Adalines 
with  a  weighting  factor  w^  for  each  pattern  vector  component  jq,  a  summing  device  for  each 
pattern  class  plus  a  maximum  selector.  Training  is  performed  by  one  adjustment  to  the  classifier 
weight  vectors  for  each  stimulus/reaction  pair  presented. 


Fig  5:  A  linear  machine  for  classifying  X  patterns  with  d  components  each  to  R 
categories  (Nilsson  1965,  modified ). 

Following  this  short  description  of  two  connectionist  learning  mechanisms  possible  applications 
of  these  concepts  to  implement  adaptive  user  interfaces  are  reviewed  and  discussed.  This 
concents  interactive  visual  pattern  classification  (Kraiss  1982),  intelligent  display  control 
(McCandless  1986 )  and  the  adaptive  training  of  a  controller  (Guez  &  Selinski  1988). 
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4.  Examples  for  adaptivity  in  man  machine  systems 

* 

Interactive  visual  Bittern  classification  (Krais*  1982) 

Classification  of  visual  patterns  is  a  task  that  only  in  a  few  cases  can  be  fully  automated.  This  is 
mainly  due  to  the  fact  that  many  of  the  criteria  applied  by  an  observer  can  not  be  quantified  and 
stated  explicitly.  What  one  sees  depends,  e.g.,  on  what  one  looks  for  or  expects  to  see,  the 
context  and,  last  not  least,  the  costs  of  not  seeing  it  Expert  knowledge  and  long  term 
professional  experience  will  have  a  major  influence  on,  e.g.,  the  classification  of  targets  from 
electro-op  deal  sensor  systems  or  on  the  extraction  of  features  from  x-ray  pictures.  This  case 
study  addresses  the  question  whether  computer  aiding  can  be  helpful  m  improving  classification 
consistency  of  human  observers.  The  term  "consistent”  implies  that  a  particular  pattern,  if 
presented  several  times,  is  always  assigned  to  the  same  category.  A  system  concept  for  this  study 
incorporates  an  adaptive  observer  that  continuously  traces  the  visual  patterns  presented  to  the 
operator  as  well  as  his  choice  and  learns  implicitly  the  decision  strategy  of  the  human  (compare 
fig.  3).  The  need  to  quantify  and  state  explicidy  the  complex  criteria  applied  by  the  expert  user  is 
thus  avoided. 

The  adaptive  observer  will,  after  sufficient  training,  suggest  a  choice  that  is  in  line  with  observer 
preferences.  Sometimes,  the  human  will  be  confronted  with  a  contradicting  proposal  indicating 
that  his  decisions  have  not  been  consistent  Even  in  that  case,  however,  he  is  entirely  free  to 
make  up  his  mind  which  will  eventually  result  in  a  retraining  of  the  adaptive  device.  The  trainable 
observer  may  thus  be  seen  as  a  intelligent  monitor  watching  the  consistency  of  observer 
decisions  and  forcing  the  human  to  reconsider  classifications  which  do  not  fit  in  with  previous 
actions. 

For  the  evaluation  of  the  described  system  concept  a  set  of  50  visual  patterns  was  generated  at 
random.  Each  partem  is  composed  of  20  columns  with  varying  heights.  A  particular  pattern  i 
may  be  described  by  a  pattern  vector  Xj*[xlljtu^..jt0o]  where  each  vector  component  may 
assume  random  values  with  -1  <Xg<+l  (see  figure  6,  lower  pan).  For  reference,  patterns 


Fig. 6  Experimental  setup.  Test  patterns  ki  the  middle  of  the  screen 

must  be  assigned  to  one  of  6  candidate  classes  on  top. 
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which  ire  known  to  belong  to  categories  1  -  6  are  displayed  on  top  of  the  screen.  The 
mle  m  he  performed  is  the  assignment  of  a  pattern  to  cm  of  the  available  candidate 
r.iasres,  The  adaptive  observer  was  for  this  work  selected  to  be  a  linear  machine  as  depicted  in 
figure  5. 

Two  groups  of  6  subjects  were  used  in  an  experiment  Each  subject  had  to  assign  six  series  of  SO 
visual  patterns  to  6  candidate  classes.  Thus  300  individual  decisions  were  collected  from  each 
subject  During  the  experiments  subjects  were  instructed  to  make  their  choice  solely  according  to 
the  criterium  ’similarity''  and  concentrate  on  the  ’Gestalt".  No  cost  risk  or  time  pressure  for 
decision  making  was  imposed.  Reference  to  a  physical  background  for  the  patterns  was  strictly 
avoided,  therefore  no  special  background  knowledge  was  requested  and  no  experts  were  needed 
to  participate  in  the  test  runs. 

Without  aiding  the  subject  had  to  press  one  out  of  six  buttons  to  indicate  the  selected  class.  In 
case  of  aiding  the  information  presented  on  the  screen  took,  e.g.,  the  following  form: 

PROPOSED  CLASS  NO.:  I  (45  %),  V  (27  %) 

RELIABILITY:  60  % 

PLEASE  TYPE  IN  THE  SELECTED  CLASS  NO.: 

The  priorities  of  computer  proposals  and  the  appertaining  probabilities  have  been  calculated  using 
the  actual  values  for  the  6  discriminant  functions  g;  (X)  of  the  adaptive  observer  (compare  fig. 
5).  Only  the  two  most  probable  candidate  classes  are  displayed  to  the  observer  in  order  to  avoid 
confusion.  The  line  "Reliability"  indicates  how  often  aiding  was  successfully  accepted.  The 
indicated  number  is  a  sliding  average  over  the  last  10  patterns.  In  case  of  inconsistent  operator 
decisions  the  computer  find*  out  which  classifications  are  in  conflict.  In  such  cases  the  operator 
is  aware  of  his  own  conflicting  decisions  by,  e.g.,  the  following  text: 

PLEASE  CONFIRM  YOUR  CHOICE  (CLASS  NO.  3  OR  6  ?): 

The  answer  to  this  question  is  taken  as  the  operators  final  :hoice.  Conflicting  answers  made 
earlier  are  eliminated  and  substituted  correctly  so  that  at  any  instance  a  consistent  pattern  set  is 
available  for  training. 

Data  from  both  experimental  groups  show  that  only  for  very  few  patterns  subjects  made  identical 
choices.  Most  patterns  have  been  assigned  to  several  classes  (up  to  four).  Since  subjects  had  to 
classify  6  identical  pattern  sequences,  individual  consistency  can  be  determined  by  comparing 
class  assignments  for  corresponding  patterns  in  subsequent  sequences.  These  calculations  have 
been  performed  for  the  aided  as  well  as  for  the  unaided  group.  The  values  given  in  figure  7  for  a 
particular  pattern  sequence  indicate,  how  often  a  classification  was  selected  that  was  not  in 
agreement  with  the  one  made  in  the  preceding  sequence. 
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Fig.  7:  Individual  classification  inconsistency  with  and  without  aiding 
(data  from  two  groups  of  6  Ss  cadi). 

Without  aiding  the  individual  inconsistency  starts  in  the  range  of  about  30  percent.  During 
subsequent  training  sequences  some  improvement  can  be  observed  with  a  tendency  to  level  off 
above  20  percent.  The  situation  is  markedly  better  when  aiding  is  introduced.  In  this  case 
performance  starts  at  a  lower  level  of  about  16  percent  inconsistency.  There  seems  to  be  a  steady 
stabilization  effect  during  subsequent  runs  resulting  in  a  classification  inconsistency  as  low  as  3 
%  for  the  6th  patten  sequence.  Standard  deviations,  which  are  also  given  in  this  figure,  indicate, 
that  this  very  low  inconsistency  level  is  stable  among  subjects. 


Intelligent  disnlav  control  (McCandless  1986) 

Along  with  the  continuously  increasing  complexity  of  man  machine  systems  it  has  long  been 
recognized,  that  operators  can  not  be  presented  with  all  available  information  without  suitable 
filtering  and  integration.  Considerable  effort  has  been  made  in  order  to  reduce  the  amount  of 
information  at  the  user  interface  by  situation  and  task  dependent  filtering.  In  modem  airliners, 
e.g.,  adaptation  to  particular  phases  of  operation  is  provided:  only  such  information  is  presented 
or  activated  in  the  cockpit  which  is  necessary  and  useful  during  taxiing,  take-off,  cruising  or 
landing  respectively.  Another  approach  has  been  to  facilitate  information  processing  and  flow  by 
the  suitable  integration  and  display  of  distributed  pieces  of  information.  So  far  little  effort  has 
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been  made,  to  dynamically  adapt  the  interface  to  individual  preferences  and  variations  of 
operators  skills. 

The  connectionist  model  proposed  by  McCandless  is  designed  to  adaptively  control  the  display 
of  icons,  rfi»gr«m  or  windows.  An  adaptive  observer  monitors  the  system's  state  and  provides 
an  intelligent  organization  of  displayed  information  (fig.8).  Inputs  to  the  network  are  data 
collected  on  system  variables,  which,  after  suitable  statistical  treatment  are  fed  to  sensor  units. 
Sensor  units  discriminate  three  states  for  each  variable  (increasing,  decreasing,  domant  ).The 
icon  control  units  gather  input  from  every  sensor  and  from  every  other  icon  control  unit.  The 
within  level  links  provide  competition  for  presentation  among  icons.  The  network  units  on  the 
icon  control  level  pass  their  activation  on  to  a  set  of  diagram  control  units.  This  allows  the 
controller  to  identify  and  present  currently  important  icons  and  diagrams  to  the  user. 


Figure  8.  Basic  control  layout  for  intelligent  display  control  (McCandless  1985). 

Again,  instead  of  extracting  knowledge  from  an  expert,  the  network  automatically  and 
continuously  records  the  behaviour  of  an  expert  using  the  system  in  a  particular  situation.  The 
resulting  activations  of  the  network  output  units  represent  the  importance  of  a  pieces  of 
information  needed  for  diagnosis.  In  general  several  units  are  competing  with  each  other  for  the 
use  of  limited  screen  control  space. 

In  the  course  of  operation  the  connectionist  display  controller  learns  about  normal  sequences  of 
system  states  which  occur  and  about  user/expert  criteria  for  information  coding  and  organisation 
on  the  display.  Currently  a  version  of  this  controller  has  been  implemented  as  a  control 
mechanism  for  organising  the  icons  and  diagrams  displayed  in  STEAMER,  a  system  used  to 
train  the  operation  of  a  steam  propulsion  plant  ( Hollan  et  aL  1984 ).  Operational  experience  has 
not  been  reported  yet 


The  network  described  above  is  limited  to  interpreting  the  current  importance  of  icons  and 
diagrams.  It  is  however  also  possible  to  predict  regularly  occuring  sequences  of  system  states 
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using  an  additional  connectionist  network.  The  predictor  information  may  then  be  used  to 
provide  the  operator  with  advance  information  on  future  diagnostic  tasks. 


Adaptive  training  of  a  controller  (Guez  &  Selinsky  1988) 

Trainable  adaptive  controllers  are  a  subset  of  process  controllers  where  the  design  is  done  by 
on-line  teaching  instead  of  off-line  control  theoretical  computations.  While  in  both  application 
examples  presented  above  the  adaptive  observer  was  trained  to  leant  discrete  events,  we  now 
address  the  continuous  case.  The  basic  idea  however  remains  the  same  (fig.9):  The  adaptive 
controller  learns  a  suitable  control  strategy  by  observing  a  human  teacher.  After  being  sufficiently 
well  trained,  the  neural  netwok  can  take  over  and  duplicate  the  behaviour  of  the  human.  The 
trainer  may  then  be  removed  or  he  may  remain  standby  as  a  monitor.  Retraining  and  manual 
takeover  is  possible  at  any  time  the  operator  wishes. 


Figure  9:  Trainable  adaptive  controller  architecture 

Currently  this  approach  has  been  successfully  tested  far  a  cart-pole  system  (the  network  chosen 
for  the  simulation  has  2  hidden  layers,  4  input  neurons  and  1  output  ).The  results  show  that  the 
system  was  able  to  leant  and  generate  stable  control  from  the  examples  generated  by  a  human 
teacher.  This  demonstrates  that  the  control  law  could  be  identified  without  being  explicitly  stated. 


5.  Conclusions 

This  paper  addressed  different  aspects  of  adaptivity  in  man  machine  systems.  An  architecture  for 
cooperate  man  machine  systems  for  this  purpose  was  outlined.  Adaptive  filters  and  neural 
networks  were  mentioned  as  mechanisms  to  implement  computer  learning.  Three  case  studies 
were  presented,  which  demonstrated  interactive  visual  pattern  classification,  intelligent  display 
control  and  adaptive  controller  training. 
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From  this  it  appears  that  the  neural  net  approach  could  be  an  elegant  solution  to  knowledge 
acquisition  a«in  which  currently  still  is  a  very  serious  and  basically  unsolved  problem  of  rule 
based  expert  systems.  Hens  it  is  sufficient  to  observe  an  experts  behaviour  instead  of  asking  him 
questions.  Consequently  there  is  no  need  to  explicitly  formulate  a  set  of  rules.  There  is  no  need 
to  explicitly  update  and  modify  of  rules  if  tasks  or  situations  have  changed.  After  successful 
training  of  a  net  the  learned  knowledge  resides  in  the  linking  weights  of  die  neurons.  From  these 
it  is  easy  to  see,  which  inputs  were  considered  essential  by  an  operator  and  which  were 
neglected. 

From  the  reported  studies  it  appears  that  adaptivity  at  the  user  interface  can  result  in  improved 
individual  decision  consistency  in  pattern  recogition  tasks,  can  support  diagnostic  tasks  by 
providing  suitable  information  display  and  can  duplicate  human  manual  control  behaviour. 
Further  studies  axe  needed  to  work  out  this  approach  in  more  detail  and  to  test  it  in  an  operational 
context 
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MULTI-MAN  CfleWSTATION  DESIGN  IN  AN  IKBS  TECHNOLOGY 
DEMONSTRATOR  FOR  THE  ROYAL  NAVY. 

Geoige  Brander 

Ministry  of  Defence,  Procurement  Executive 
Admirely  Research  Establishment 
Portsdown,  Portsmouth  P06  4AA 

Summary 

A  Technology  Demonstrator  Programme,  in  progress  at  the  Admiralty  Research 
Establishment  (Portsdown),  aims  to  explot  and  explore  the  benefits  of  Intelligent  knowledge- 
based  systems  technology  (IKBS)  in  the  area  of  real-time  Naval  command  and  control.  This  paper 
outlines  the  features  of  the  demonstrator  and  discusses  the  objectives  of  the  programme, 
together  with  some  of  the  key  issues  arising. 

Introduction. 

Over  the  past  few  years  research  effort  at  the  Admiralty  Research  Establishment  has  been 
examining  the  use  of  knowledge-based  programming  techniques  in  the  domain  of  Nava)  command 

and  control.  The  research  has  produced  a  laboratory  prototype  which  has  successfully  proven  the 
validity  of  the  technical  concepts.  The  work  now  continues  under  a  Technology  Demonstrator 
Programme,  the  purpose  of  which  is  to  produce  a  sea-going  demonstrator  which  can  be  trialled 
and  evaluated  In  an  operational  setting  during  the  early  1990's. 

Data  Fusion. 

The  main  thrust  of  the  research  to  date  has  concentrated  on  the  area  of  data  fusion,  that  is 
the  completion  of  a  tactical  picture  which  wH  present  the  command  team  with  a  concise,  accurate 
and  comprehensible  representation  of  the  tactical  situation  facing  them. 

Todays  warships  receive  an  ever  increasing  volume  of  information  from  a  variety  of  sensors, 
both  organic  and  non-organic.  which  must  be  assimilated,  interpreted  and  assessed.  In  order  to 
achieve  an  underatancSng  of  the  external  world,  ft  is  necessary  to  detect,  locate,  track  and,  if 
possible,  classify  all  the  objects  which  might  conceivably  contribute  to  the  tactical  situation.  This 
knpaes  virtually  every  object  within  the  sensor  range  or  within  the  volume  of  interest  of  a  single 
warship  or  of  a  group  of  co-operating  maritime  units,  which  may  be  dispersed  over  a  wide  area  of 
ocean.  Information  sources  Include  plans  and  objectives,  radio  datalinks,  acoustic  and  optical 
devices,  human  observers  providing  intelligence  data,  as  well  as  the  more  dynamic  real-time 
sensor  Information.  The  task  of  combining  such  disparate  data  types  has  proved  to  be  beyond  the 
capabilities  of  conventional  computing  methods,  yet  it  has  remained  the  province  of  the  already 
overloaded  human  operator,  even  though  it  has  to  be  undertaken  within  the  very  short  timescales 
dictated  by  the  speed  of  modem  warfare. 

The  techno  logical  solution  selected  for  the  data  fusion  demonstrator  utilises  a  rule-based 
approach  to  generate  a  hierarchy  of  hypotheses.  This  has  been  implemented  using  a  blackboard 
type  of  expert  system  architecture.  Further  rules  are  applied  to  select  the  ’current  best  view’,  that 
Is  the  most  likely  hypothesis,  which  Is  presented  to  the  operator.  Correlations  not  selected  are 
retained,  however,  m  case  any  choices  should  prove  incorrect  following  the  arrival  of  more  data. 
This  method  represents  multipie  hypotheses  at  a  low  level  but  generates  only  one  conclusion  in 
order  to  avoid  confusing  the  operator  with  a  combinatorial  explosion  of  potential  solutions.  Should 
this  conclusion  prove  incorrect,  it  is  possible  to  refer  back  to  the  lower  levels  to  generate  a  new, 
consistent  solution.  For  further  details,  see  References  i  and  2. 
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Situation  Aaaaaamant. 

AShough  data  fusion  generates  a  representation  of  the  tactical  world,  this  representation  is 
essentially  low  level  and  further  inferences  are  required  to  provide  the  Information  on  which 
decisions  must  be  made.  This  extension  is  referred  to  as  situation  assessment,  where  the 
emphasis  is  on  provWng  an  assessment  of  the  implications  of  the  perceived  world.  Thus  elements 
of  the  tactical  picture  may  be  combined,  for  example,  formations  of  hostile  aircraft,  in  order  to  infer 
the  type  and  strength  of  a  potential  threat.  At  a  higher  level  still,  elements  of  an  opponent's  tactical 
plan  may  be  identified  which  may  be  used  to  infer  the  missions  of  unknown  units  whose  presence 
in  the  tactical  picture  was  previously  unexplained. 

Resource  Allocation. 

The  next  stage  m  the  command  and  control  process  is  the  response  to  the  perceived  tactical 
situation.  This  is  the  reactive  part  of  the  system  and  is  referred  to  as  resource  allocation.  This  term  is 
rather  imprecise,  however,  as  the  reaction  may  include  the  detach  mem  of  subordinate  assets, 
such  as  combat  air  patrol  fighters  to  counter  a  long  range  air  attack,  as  well  as  the  Immediate 
response  to  the  detection  of  a  submarine  launched  torpedo  at  short  range.  Work  Is  progressing  on 
this  type  of  decision  support  facility  in  order  demonstrate  the  capability  of  an  infeiSgem  knowledge- 
based  systems  approach  to  the  whole  range  of  command  and  control  activities. 

Human-Computer  Interface. 

In  order  to  make  the  facilities  described  above  available  to  the  user,  special  purpose  graphics 
software  has  been  developed  to  drive  a  high-resolution  colour  graphics  terminal  using  the  GKS 
protocols.  Multiple  logical  windows  can  be  created  which  allow  the  user  to  examine  the  different 
levels  of  the  system,  Inducing  the  blackboard  data  structure.  There  are  two  main  types  of  window: 
plan-view  windows,  which  show  a  graphical  view  of  the  tactical  picture  and  text-windows,  which 
allow  alphanumeric  data  on  elements  ot,  or  objects  within,  the  tactical  picture  to  be  examined. 
Figure  1  shows  an  example  ot  the  human  computer  Interface  envisaged  at  this  stage  of  system 
development.  Manipulation  of  the  display  facilities  is  accomplished  by  means  of  a  hierarchical  menu 
system. 

In  addition  a  range  of  explanation  facilities  has  been  developed  (Reference  3).  First  there  is 
a  textual  window  which  displays  the  hypothesis  concerning  a  selected  object  in  the  tactical  picture, 
as  we«  as  listing  those  hypotheses  which  support  »  at  tower  levels  and  that  which  Is  supported  by  it 
at  a  higher  level.  Next  there  is  a  textual  justification  which  declares  the  specific  conditions  which 
have  been  applied  and  the  particular  parameters  which  support  the  conclusion  being  queried. 
Finally  there  is  a  graphical  representation  of  the  hypothesis  network  connected  to  a  selected 
hypothesis,  which  quickly  shows  the  user  the  evidence  supporting  an  object  In  the  fused  picture. 

Objectives  and  Problems. 

What  then  are  the  main  objectives  of  this  Technology  Demonstrator  Programme  and  what 
are  the  issues  and  problems  arising  from  them? 

i .  Doss  the  technology  work? 

At  the  level  of  data  fusion  this  is  relatively  straightforward.  Does  the  tactical  picture  produced 
by  the  data  fusion  system  match  or  improve  upon  the  tactical  picture  complied  by  more 
oonventkmsi  manual  command  teams  In  terms  of  speed  and  accuracy?  Exercise  analysis 
techniques  already  exist  to  evaluate  the  performance  of  command  teams  by  comparing  their 
perceived  world,  at  contained  within  the  ship's  command  system  computer,  with  the  evidence  ot 
what  actually  happened  in  the  real  world,  as  re-constructed  from  exercise  plans  and  detailed 
recordbiga. 
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Evaluation  of  situation  assessment  and  resource  allocation  functions  becomes  more 
subjective  as  these  involve  judgement  and  decision  making  on  behalf  of,  or  in  conjunction  with, 
the  expert  user.  Indeed,  because  the  system  is  dealing  with  heuristic  knowledge,  there  Is  the 
impfication  that  there  may  be  no  formal  proof  that  a  given  result  or  solution  is  correct.  Naval 
Exercises  will  provide  the  operational  setting  within  which  the  demonstrator  wil  be  trialled  and 
current  practice  in  these  exercises  Involves  expert  observers  evaluating  the  performance  of 
human  operators  in  realistic  combat  conditions,  at  sea.  It  is  envisaged  that  a  similar  mechanism 
should  alow  assessment  of  the  demonstrator  system  responses. 

2.  What  can  we  learn  about  how  best  to  procure  and  exploit  intelligent 

knowledge-based  systems? 

Methods  of  specification,  validation  and  acceptance  for  real-time  knowledge-based  systems 
are  virtually  non-existent  The  demonstrator  will  provide  a  suitable  environment  to  undertake  initial 
experimentation  into  these  topics.  Many  issues  will  be  raised  in  the  fields  of  software  engineering, 
knowledge  engineering  and  knowledge-based  systems  technology.  Our  experiences  in  the 
design,  implementation  and  evaluation  of  the  demonstrator  will  establish  a  solid  framework  on 
which  recommendations  and  procedures  for  the  successful  procurement  of  command  systems  for 
the  next  generation  of  Royal  Navy  ships  can  be  based. 

There  are  many  problems  In  the  area  of  requirements  specification  tor  complex  systems,  but 
especially  so  for  intelligent  knowledge-based  systems.  The  sub-systems  of  the  laboratory 
prototype  are  based  upon  a  technological  model  of  command  and  control.  The  Naval  authority 
responsible  for  generating  requirements  for  operational  command  and  control  systems,  however, 
utilise  a  user-oriented  model.  The  several  thousand  functions  produced  by  this  latter  method, 
although  they  assume  no  allocation  of  function  decisions  or  implementation  solutions,  do  not 
approach  the  level  of  detailed  knowledge  required  by  the  designers  of  intelligent  functions. 
Current  methods  of  requirements  capture  seem  to  identify  explicit  steps  and  procedures  but  do 
not  readily  represent  the  impficit  knowledge  and  rules  contained  therein.  It  is  precisely  this  implicit 
knowledge  that  must  be  made  explicit  in  the  implementation  of  knowledge-based  systems. 

In  addition,  it  may  be  argued  a  knowledge-based  system  cannot  fully  be  specified  except  by 
defining  the  entire  rule-base.  The  implication  is  that  a  highly  detailed  specification  of  the  rule-base 
must  be  produced  before  the  operational  system  can  be  procured. 

Further  problems  are  envisaged  in  the  management  of  the  roles  and  knowledge  contained 
within  intelligent  knowledge-based  systems.  Should  the  operational  user  be  allowed  to  tamper 
with  the  knowledge-base  or  role-base  during  the  course  of  a  mission  if  he  learns  new  facts  or 
develops  new  inference  roles?  This  may  be  technically  possible  with  these  new  systems,  but  it 
would  mean  that  each  system  could  be  individually  evolved.  Who  would  then  have  the  authority  to 
declare  the  system  acceptable?  There  would  seem  to  be  a  requirement  for  a  Naval  Organisation  to 
develop,  maintain  and  evaluate  developing  knowledge  and  roles  and  to  issue  periodic  updates  to 
these,  In  much  the  same  way  as  Tactical  Publications  are  developed  and  issued  today. 

Finally,  the  operational  acceptance  of  intelligent  systems  is  a  difficult  issue.  A  knowledge- 
based  system  could  be  seen  as  being  similar  to  human  operator  just  out  of  basic  training;  needing 
experience  and  on-the-job  training  to  develop  his  skills.  Would  this  imply  a  phased  acceptance 
procedure  (or  these  new  systems,  with  assessment  tests  being  applied  over  an  increasingly 
difficult  range  of  test  scenarios,  until  the  system  is  judged  satisfactory?  The  implications  of  sending 
a  'raw-recruit*,  immature  system  to  sea  have  not  been  addressed,  even  though  experience  with 
current,  conventional  command  systems  suggests  that  certain  types  of  deficiencies  quickly 
alienate  the  user  to  the  extent  that  several  years  may  pass  before  faith  in  the  system  is  established. 
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3.  How  can  the  User  Interface  with  such  Intelligent  systems  be 

designed? 

Little  is  known  about  the  design  of  interlaces  between  experts  and  Intelligent  systems 
operating  on  real-time  Information.  Issues  of  confidence  and  trust  become  important  because, 
even  though  the  system  will  allow  the  user  to  track  its  reasoning  processes  and  can  provide 
justification  of  its  conclusions,  the  timescales  within  which  responses  are  required  may  proscribe 
the  full  use  of  this  explanation  feature  during  the  stress  of  combat.Titis  implies  that  the  user  will 
need  to  develop  his  confidence  in  the  reasoning  component  of  the  machine  under  simulated  or 
exercise  conditions,  before  he  will  be  able  to  accept  its  recommendations  in  more  a  realistic 
setting.  This  process  may  be  similar  to  the  way  in  which  military  officers  already  develop  trust  and 
confidence  in  their  human  colleagues  based  upon  assessments  of  their  reliability,  judgement  and 
conviction  during  previously  shared  experiences. 

In  addition  the  user  is  likely  to  maintain  his  own  internal  model  of  the  scenario  and  may  have 
information,  not  available  to  the  system,  which  Is  vital  to  the  assessment  of  the  tactical  picture.  We 
need  to  explore  mechanisms  by  which  the  user  can  interact  with  the  machine's  reasoning  process 
and  override  or  enhance  the  evidence  with  his  own  endorsements  or  explanations.  This  is  likely  to 
be  extremely  difficult  if.  as  suggested  above,  the  speed  of  modem  warfare  does  not  permit  the 
user  enough  time  to  engage  in  a  dialogue  with  the  machine  as  to  why  he  knows  that  object  X  is  not 
hostile  when  the  machine  thinks  9  is. 

Problems  In  Crewstatlon  Design. 

In  adcWon  to  the  issues  discussed  above  there  Is  considerable  interest  in  the  implications  for 
reduced  manning  resulting  from  this  new  technology.  Manpower  represents  a  large  proportion  of 
the  cost  of  a  fighting  ship,  despite  the  ever  increasing  cost  of  ship  systems.  As  systems  become 
more  complex,  training  becomes  both  more  critical  and  costly.  Current  Naval  operations  rooms 
employ  between  20  apd  40  personnel,  depending  upon  the  size  and  role  of  the  ship.  These 
operators  range  from  Able  Seamen  (Junior  Rates)  up  to  Senior  Officers.  Although  they  are  divided 
into  small  teams  with  specific  responsibilities,  such  as  sensor  monitoring  or  weapon  control,  many 
of  these  operators  are  engaged  in  activities  which  correspond  to  the  proposed  technical  functions 
of  data  fusion,  situation  assessment  and  resource  allocation.  However,  these  activities  do  not 
occur  in  a  neat  sequential  way,  they  occur  continually  and  cyclically,  in  several  locations  and 
dispersed  amongst  various  operators.  The  introduction  of  new  technology  will  effectively  automate 
some  of  the  lower  level  functions  performed  by  the  more  junior  operators  but,  although  this  may 
reduce  the  manpower  required,  it  will  also  have  considerable  impact  upon  task  design,  team 
organisation,  training  and  career  development.  In  addition,  many  of  these  operators  have  other 
less  operational  jobs  on-board  such  as  ship  husbandry,  manning  boat  parties  and  damage  control. 

The  problem  of  allocation  of  function  between  man  and  machine  has  been  recognized  but 
has  resisted  rigorous  solution  for  some  thirty  years.  Job  design  and  team  organisation  have 
similarly  made  little  progress.  This  is  probably  because  these  issues  are  not  simply  stages  in  the 
design  process  but  rather  the  design  itself.  The  manipulation  of  trade-offs  between  conflicting 
requirements  must  be  seen  In  the  context  of  the  through-fife  costs  of  the  system  and  must 
address  the  issues  of  additional  training  needs,  recruitment  and  retention,  job  satisfaction  and  all 
the  other  socfo-technicai  concepts  which  lack  rigorous  methods  and  procedures. 

Current  command  teams  on-board  ships  often  adopt  flexfcle  working  procedures.  When  one 
operator  appears  to  be  overloaded  another  may  dose  up  and  reduce  his  load  by  taking  over  certain 
functions.  Teams  are  adjusted  according  to  the  duration  and  pace  of  particular  tactical  scenarios. 
We  do  not,  I  believe, have  a  sufficient  understanding  of  how  this  dynamic  task  allocation  process 
occurs  and  should  devote  more  effort  to  developing  techniques  both  to  model  It  and  to  evaluate 
the  alternative  solutions. 
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Conclusion. 


T 


The  advent  of  knowledge  based  systems  technology  will  offer  considerable  advantages  in 
the  field  of  Naval  command  and  control,  although  It  raises,  as  was  ever  the  case,  significant 
problems  in  Its  introduction.  The  technology  demonstrator  system  under  development  at  ARE 
offers  the  opportunity  to  explore  some  of  the  fundamental  issues  raised  by  the  technology.  It  is 
hoped  that  the  inherent  flexibility  of  the  technology,  in  the  sense  that  it  may  enable  a  'rapid 
prototyping  approach',  will  allow  new  ideas  to  be  introduced  in  an  evolutionary  way  as  experience 
and  feedback  are  gained  from  the  use  of  the  system  in  an  operational  environment. 
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SUMMARY 

The  paper  describes  a  ground  based  mission  planning  demonstrator  system  developed  by 
^  means  of  an  IKBS  workstation  and  discusses  the  evolution  of  the  concept  into  an  1n- 

(  flight  planner.  The  concept  of  an  Intelligent  Displays  Manager  is  then  introduced 

>  and  the  architecture  of  a  model  under  development  is  described. 

The  relationships  between  Planner,  Olsplays  Manager  and  the  man-machine  interface 
t  cast  useful  light  on  the  problems  and  approaches  Inherent  in  realising  an  Integrated 

f  electronic  crew  member. 

I  1  INTRODUCTION 

Automated  Mission  Planning  offers  the  potential  for  a  greatly  reduced  pilot  workload 
and  increases  in  the  effectiveness  of  an  aircraft.  Intuitively,  people  will  probably 
,  agree  with  this;  however,  detailed  knowledge  of  the  structure  of  Mission  Plans 

allows  support  in  another  area.  An  Expert  System  with  information  on  Mission  Plans 
is  a  cornerstone  of  the  future  interaction  between  crew  members  and  the  electronic 
cockpit. 

The  Flight  Automation  Research  Laboratory,  FARL,  of  GEC  Avionics  have  looked  at 
Expert  Systems  both  in  Automated  Mission  Planning  and  in  crew-to-cockpit 
interfacing.  The  Expert  Systems  are  both  workstation  based  prototypes  or 
demonstrators,  but  they  show  the  feasibility,  of  both  concepts.  This  paper  will 
briefly  describe  our  work  in  this  area,  then  it  looks  at  the  next  step  forward;  in¬ 
cockpit  Expert  Systems. 

2  MISSION  PLANNING 

The  Mission  Planning  work  concentrates  on  one  area;  that  of  Long  Range  Ground 
Interdiction.  This  area  can  be  considered  in  many  different  ways  depending  upon 
ones  viewpoint.  That  is.  Mission  planning  requires  expertise  from  many  different 
areas.  A  list  of  possible  experts  for  Long  Range  Ground  Interdiction  follows: 

0  Threat  avoidance  0  Tactics  for  several  aircraft 

0  Stealth  0  Tatics  for  weapons  delivery 

0  Waypoint  selection  0  Navigation 

2.1  Route  Planning 

Flight  Planning,  in  the  context  of  this  work,  is  related  to  a  pilot  selecting  a 
route  to  a  target  on  a  Long  Range  Ground  Interdiction  mission.  It  is  essentially  a 
Route  Planning  task,  and  it  is  a  current  NATO  constraint  that  only  20  minutes  are 
allocated  for  this  planning  task. 

The  pilot  selects  a  route  from  his  home  airfield;  across  friendly  territory,  over 
the  Forward  Edge  of  Battle  Area,  FESA;  through  hostile  areas  to  the  target.  The 
route  then  returns  him  over  hostile  territory;  through  friendly  areas  to  a  suitable 
airfield.  Each  area  of  this  route  requires  a  pilot  to  plan  in  a  different  way,  that 
is,  to  use  different  expertise.  The  expertise  can  be  divided  into  three  areas:  pre 
1  FEBA;  between  the  IP  and  target  and  the  remaining  route  over  hostile  territory. 

Nots  Ths  opinions  In  tuts  pspsr  ars  thoss  ot  ths  authors  and  do  not  nsestsarlly  raprassnt  those  ot 
SEC  Avionics  Ltd. 
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Th«  route  from  the  airfield  to  the  target  starts  behind  friendly  territory  and  the 
pilot  selects  a  safe  air  corridor  and  files  from  the  airfield  until  he  crosses  the 
FEBA. 

When  over  the  FEBA  the  aircraft  files  a  route  between  preselected  landmarks  called 
waypoints.  The  waypoints  are  selected  to  provide  a  route  that  skirts  around 
threatened  areas  and  minimises  the  chances  of  detection.  The  plan  should  also 
provide  a  pilot  with  a  route  within  his  fuel  and  time  limits! 

Pilots  use  a  point  some  45  -  60  seconds  before  the  target  called  a  Target  Initial 
point  or  IP.  This  Is  used  to  provide  an  accurate  run-in  to  the  target  In  terms  of 
track  and  time.  To  achieve  this  pilots  often  overfly  the  IP.  The  route  from  the  IP 
to  the  target  Is  straight  and  is  often  planned  separately  from  the  rest  of  the 
route.  This  planning  uses  a  larger  scale  map  and  much  more  detailed  Information 
about  lanctoiarks  in  the  vicinity  of  the  target. 

2.2  GEC  Avionics1  Route  Planning  Oemonstrator 

The  Route  Planning  Aid  Oemonstrator,  RPA,  has  concentrated  on  expertise  In  one  of 
the  three  areas  of  the  route;  the  route  over  hostile  territory.  The  RPA  plans 
routes  from  the  FEBA  to  the  IP,  the  return  journey  requires  identical  expertise. 

The  RPA  selects  visible  waypoints  that  minimise  the  exposure  to  known  threats  In  the 
area.  It  uses  the  same  skills  as  a  navigator,  or  pilot  would  In  selecting  the 
route. 

The  Route  Planning  Problem  was  split  Into  two  different  expert  systems.  A  Feature 
Extraction  Expert  and  a  Route  Planning  Expert. 

Although  experts  for  planning  have  been  investigated  in  many 

different  areas,  planning  is  a  word  with  very  diverse  meanings  and  it  was  not 
possible  to  apply  any  existing  planning  techniques  to  this  application. 

As  a  demonstrator  the  Route  Planning  Expert  shows  the  powerful  capabilities  of 
relatively  simple  rule  bases.  The  experts  are  composed  of  four  sets  of  rules: 
Waypoint  Selection  Rules,  Search  Control  Rules,  Selection  Rules  and  Evaluation 
Rules. 

The  RPA  expert  system  is  best  described  by  a  diagrammatic  representation. 

Search  Rules.  The  search  strategy  uses  diamond  shaped  search  areas  based  on  two 
points.  Initially  the  FEBA  exit  point  and  the  IP  (see  fig  1). 

The  search  strategy  is  recursive.  The  breakdown  of  the  problem  after  the  first 
recursion  can  be  seen  by  the  three  search  diamonds  in  fig  l.  The  large  diamond 
shows  the  first  search  area  and  the  two  smaller  diamonds  the  two  next  areas  to  be 
searched. 

Threat  avoidance  can  be  accomplished  at  the  search  level  by  modifying  the  search 
diamond  to  avoid  threats,  see  fig  2.  At  this  stage  the  features  are  selected  by  the 
Selection  Rules. 

Selection  Rules.  The  Selection  Rules  are  applied  to  the  features  returned  by  the 
Search  Rules.  This  ruleset  is  also  responsible  for  threat  avoidance  and  rejects  any 
features  lying  In  a  threatened  area;  features  A  and  B  In  fig  l.  It  then  applies 
further  rules  to  select  the  best  of  the  remaining  features. 

Evaluation  Rules.  The  final  set  of  rules  within  the  RPA  are  the  Evaluation  Rules. 
These  are  the  primitives  of  the  RPA  expert  system  and  are  used  to  “classify"  the 
features.  These  classifications  are  then  used  by  the  Selection  Rules  to  select  the 
best  feature  and  to  determine  whether  a  feature  lies  in  a  threatened  area. 

Waypoint  Selection  Rules.  The  Feature  Extraction  Expert  is  an  off-line  program  that 
extracts  Flight  Planning  features  from  a  digital  map  database.  The  Feature 
Extraction  expert  is  resonsible  for  applying  the  Waypoint  Selection  Rules.  This 
Information  Is  then  used  as  an  Information  Base  by  the  Route  Planning  Expert.  This 
provides  a  very  valuable  way  of  pre-compiling  information  required  by  the  RPA. 
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After  using  the  capabllltites  of  a  Texas  Instruments  workstation  to  construct  the 
solution  and  develop  a  thorough  understanding  of  the  problem  domain,  the  RPA  was 
successful?  re* Implemented  In  Pascal  on  an  IBM-PC.  This  demonstrated  the  feasibllty 
of  a  low  cost  solution  that  Is  both  smaller  and  faster  than  the  LISP  Implementation. 

3  MISSION  PLANS  IK  THE  CREW  TO  ELECTRONIC  COCKPIT  INTERFACE 

Intelligent  Displays  Management  Is  the  second  area  of  in-cockpit  IKBS  that  FARL  have 
addressed.  This  work  has  been  carried  out  with  the  support  of  the  Procurement 
Executive,  Ministry  of  Defence. 

Even  with  the  amount  of  avionic  equipment  In  current  aircraft,  pilots  have 
difficulty  coping  with  all  the  Information.  Their  effectiveness  as  pilots  and 
controllers  of  the  aircraft  can  be  Increased  If  some  Intelligence  Is  used  to  present 
the  Information  to  the  pilot.  One  method,  already  mentioned.  Is  to  provide 
Intelligent  systems  within  the  aircraft  .  The  second  area  Is  In  Intelligent 
Displays  Management. 

An  Intelligent  Displays  Manager  can  be  considered  as  a  filter  to  prevent  the  pilot 
being  overwhelmed  with  uneccessary  Information.  This  Is  increasingly  Important  In 
both  civil  and  military  applications.  A  familiar  example  Is  in  system  failures 
where  cascaded  errors  tend  to  drown  a  pilot  in  Information.  Knowledge  of  the 
Mission  Plan,  required  to  filter  Information,  can  be  used  to  anticipate  a  pilots 
requests  and  needs.  This  will  allow  a  future  Intelligent  Cockpit  to  reconfigure 
displays  and  represent  Information  In  a  more  appropriate  manner.  The  limitations  of 
general  display  formats  will  be  removed  and  a  whole  new  generation  of  displays  can 
be  Introduced. 

Equally  Important  Is  the  Intelligent  control  of  user  Inputs.  Here  an  Intelligent 
Displays  Manager  can  provide  context  sensitive  controls  over  the  Interpretation  of 
pilot  Inputs.  This  can  be  useful  In  understanding  the  meaning  of  a  single  button  on 
the  joystick,  providing  flexible  softkeys  or  In  assisting  in  the  Interpretation  of 
Olrect  Voice  Input,  DVI. 

3.1  Intelligent  Displays  Management 

For  an  Intelligent  Olsplays  Manager  to  be  effective  it  must  be  able  to  model  a 
pilots  mindl  More  precisely  It  must  have  the  same  knowledge  of  mission  plans  as  a 
pilot,  eg  "a  waypoint  Is  approaching  so  I  will  shortly  be  changing  track". 

In  addition  to  the  knowledge  of  Mission  Plans  the  Intelligent  Displays  Manager  must 
have  knowledge  of  what  displayed  Information  a  pilot  requires  in  each  mission  phase. 
Other  In-cockpit  expert  systems  are  similar;  requiring  knowledge  of  the  the  Mission 
Plan  and  further  domain  specific  knowledge.  Fig  3  shows  an  expert  system 
architecture  for  a  family  of  expert  systems  based  on  this  approach. 

The  Protoype  Intelligent  Olsplays  Manager  is  constructed  with  two  experts.  A 
Displays  Expert  and  and  a  State  Expert.  These  can,  in  turn,  be  broken  down  further. 
The  State  Expert  Is  composed  of  the  following  components: 

Mission  Expert  -  producing  the  global  aircraft  position  in  re’ation  to  the  Mission 
Plan  and  Coals. 

Threat  Expert  -  simplified  in  our  Prototype,  but  consists  of  components  for 
Situation  Assessment  and  Threat  Response  Tactics. 

Aircraft  State  Expert  -  the  orientation  of  the  aircraft  and  health  of  the  aircraft 
systems. 

In  normal  flying  the  Mission  Expert  provides  most  of  the  capabilities  of  the  State 
Expert,  but  when  the  aircraft  Is  threatened  or  has  system  failures  the  other  experts 
contribute. 


4.  IN-COCKPIT  MISSION  PLANNING 

In  moving  from  demonstrators  to  in-cockpit  systems  a  difficulty  is  the  limits  on  the 
Man  Machine  Interface,  MMI. 
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The  work  on  Route  Planning  has  shown  us  the  Importance  of  the  MMI  and  the  need  to 
model  and  develop  the  user  Interface  as  early  as  possible.  The  physical  Interface 
has  a  crucial  Impact  on  how  an  Expert  System  Is  used.  This  should  be  modelled 
before  the  Expert  System  and  continually  developed  hand-in-hand  with  the  Expert 
System. 

When  looking  at  our  Route  Planning  Aid  Expert  System  we  noticed  the  dual  role  It 
performs.  This  Is  also  true  of  many  other  Expert  Systems.  The  dual  role  Is  that  of 
an  advisor  and  an  Inline  Autonomous  Expert.  An  Autonomous  Expert  System  produces 
results  that  a  pilot  is  informed  of  eg  "Your  route  has  been  replanned,  follow  this 
track".  This  Is  In  the  current  domain  of  problems  we  are  familiar  with;  explaining 
an  expert's  Oeclsons.  An  advisor  requires  a  more  sophisticated  Interface.  Ideally, 
It  should  respond  to  a  pilots  "what  Ifs?"  In  the  same  way  as  a  co-pilot  or  navigator 
would.  For  example: 

Pilot:  What  If  we  save  time  by  flying  through  this  storm  cloud  ? 

Navigator:  We  will  make  up  all  our  lost  time,  and  we  won't  use  up 
any  excess  fuel . 

It  Is  the  exchange  of  Ideas  and  knowledge  along  these  lines  that  prevents  us  with 
the  most  exciting  challenge. 

5  IN-COCKPIT  DISPLAYS  MANAGEMENT 

One  of  the  most  Interesting  things  to  come  from  the  work  on  the  Prototype 
Intelligent  Olsplays  Manager  Is  the  partially  symbiotic  relationship  between  Expert 
Systems  such  as  Mission  or  Route  Planners  and  an  Intelligent  Display  Manager. 

On  the  one  hand.  In-cockpit  Expert  Systems  raise  the  level  of  communication  from 
simple  data  presentation  and  selection  to  the  exchange  of  Ideas,  or  knowledge.  It 
is  extremely  difficult  to  communicate  this  high  order  Information,  particularly  in 
the  high  stress  and  tight  real  time  conditions  of  an  aircraft  cockpit.  One  way  to 
tackle  this  Is  with  an  Intelligent  Displays  Manager. 

On  the  other  hand,  an  Intelligent  Displays  Manager  requires  information  about  the 
state  of  the  pilot's  mind.  This  requires  inputs  from  other  experts  such  as  Situation 
Asssors  and  Mission  Planners.  It  Is  these  same  experts  that  were  referred  to  in 
section  2,  where  it  was  mentioned  that  their  capabilities  could  be  used  to  reduce  a 
pilots  workload. 

If  we  are  going  to  have  Expert  Systems  in  the  cockpit  then  we  need  an  Intelligent 
Oisplays  Manager  to  make  communication  possible. 

6  DISCUSSION  PROVOCATION 

We  have  described  two  necessary  elements  of  the  electronic  crew  member.  We  wish  to 
put  to  you  questions  which  It  seems  to  us  need  to  be  addressed  when  we  come  to  their 
integration  Into  a  cohesive  man-machine  Interactive  system. 

Can  we  design  the  method  of  interaction  such  that  the  EC  can  infer  pilot  intentions 
while  at  the  same  time  reducing  Interaction  workload? 

The  knowledge  and  intention  exchanges  between  pilot  and  EC  are  going  to  represent  a 
link  of  which  we  have  to  demand  the  highest  integrity.  Do  we  know  how  to  implement 
cross-monitoring  and  reversion  strategies? 
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Summary 

The  workload  of  helicopter  pilots  can  be  characterized  by  high  visual  demands  and 
two  continually  busy  hands.  Analyzing  the  situation,  some  very  critical  tasks  can  be 
found  (especially  in  helicopters  with  only  one  pilot),  where  additional  input  facilities 
and  information  channels  are  needed.  It  was  experimentally  tested  in  a  helicopter 
simulator  which  improvements  can  be  achieved  by  integrating  voice  input,  voice 
output  and  speechfiling  systems.  Tasks  like  frequency  selection  for  communication 
channels,  voice  output  for  checklists  and  emergency  procedures  and  speechfiling  of 
flightplans,  weather  data  and  pilots'  notes  were  selected.  Recognition  results  as  well 
as  subjective  evaluations  show  that  the  voice  channel  is  a  valuable  addition  to 
existing  communication  forms,  especially  for  helicopters  with  only  one  pilot.  An 
integration  of  voice  systems  with  high  acceptability,  however,  needs  further 
improvements  of  existing  technology,  which  partially  can  be  achieved  by  using  more 
" intelligent "  structures  and  procedures.  Therefore  the  results  are  discussed  under 
special  consideration  of  the  improvements  achieveable  by  adding  "artificial 
intelligence  (At)"  to  the  system. 


1.  Introduction 

Workload  in  modern  military  aircrafts  in  critical  phases  often  exceeds  the  pilots 
capacity.  Critical  phases  are  those,  where  competitive  manual  and  visual  actions 
are  required,  like  landing  approaches  or  air-to-ground  missions.  This  is  even 
more  important  in  helicopters,  because  the  control  of  additional  degrees  of 
freedom  requires  the  continuous  use  of  both  hands  in  phases  of  take-off, 
landing  or  low  level  flying  /I -4/.  A  possible  solution  is  to  shift  some  tasks  to  the 
voice  channel.  The  essential  points  for  cockpit  applications  of  voice  input/output 
(voice  i/o)  are: 

•  the  possibility  for  simultaneous  activity  in  manual  and  visual  area, 

•  the  voice  characteristic  as  a  natural,  highly  trained  communication  form,  and 

•  the  small  amount  of  required  instrumental  area. 


2.  Experiments  with  voice  input/output 

Three  functional  areas  for  testing  voice  input/output  were  selected  in 
accordance  with  the  Heeresfliegerwarfenschule  Buckeburg  151: 

•  frequency  selection  by  voice  input, 

•  voice  controlled  voice  output  for  checklists  and  emergency  procedures,  and 

•  speech  filing  for  flightplans,  weather  data  or  pilots’  notes. 
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The  experimental  system  has  been  tested  in  a  helicopter  simulator.  Noise  back¬ 
ground,  movements  and  missions  are  equivalent  to  real  flight  tasks.  The  simu¬ 
lator  serves  for  instrumental  flight  training.  Visual  flight  simulation  is  not  possi¬ 
ble.  In  addition  to  the  simulator  experiments,  voice  i/o  has  been  tested  in  a  real 
helicopter,  ready  to  start  on  airfield. 


^  2.1  Frequency  selection  bv  voice  input 

[  Actual  frequency  selection  in  helicopters  is  done  by  turning  rotary  switches  at 

L  the  middle  console  for  each  digit  (or  digit  group).  Switching  attention  to  this 

B  console  combined  with  the  pilots  movement  may  lead  to  critical  flight  situations. 

\  Therefore  frequency  selection  by  voice  input  could  improve  pilots'  safety.  Instead 

'  of  setting  digits  for  a  specified  frequency  the  pilot  has  to  speak  the  name  of  the 

L  radio  station.  These  names  often  are  easier  to  remember  than  the  corresponding 

,  frequencies.  (Manual  frequency  adjustment  by  switches  as  a  backup  solution  is 

i  possible,  too.) 

f  Eight  pilots  took  part  in  the  experiments.  Each  pilot  had  to  set  up  92  frequencies. 

When  the  system  rejects  the  input  or  shows  a  recognition  error,  pilots  had  order 
to  repeat  the  utterance  once.  Table  1  shows  the  recognition  rate  mean  values. 
Frequency  selection  by  voice  was  evaluated  to  be  very  positive  and  a  realization 
would  be  welcomed  by  the  pilots. 


Recognition  rate 
mean  values 
in  % 

correctly 

recognized 

rejected 

utterances 

recognition 

errors 

in  quiet 
environment 
—  58  db(A) 

94.7 

2.1 

3.2 

with 

helicopter 

noise 

92.9 

5.2 

4 

with  helicopter  noise 
and 

flight  mission 

90.2 

8.1 

1.7 

on  airfield 
(only  1  pilot) 

92.4 

3.8 

3.8 

Table  1 


One  problem,  however,  is  the  number  of  recognition  errors,  which  is  not 
tremendously  high,  but  must  be  handled  by  the  pilot  in  such  a  way  that  his 
workload  usually  is  decreased  by  using  voice,  but  highly  increased  in  the  case  of 
rejection  or  substitution  of  commands. 


2.2  Voice  controlled  voice  output  for  checklists  and  emergency  procedures 

Checklists  and  emergency  procedures  in  helicopters  are  available  in  small  book¬ 
lets.  They  must  be  processed  in  the  given  sequence.  Especially  for  emergency 
procedures  during  flight  mission  manual  handling  of  the  booklet  increases  the 
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actual  danger.  Using  voice  controlled  voice  output  for  checklists  and  emergency 
procedures  can  avoid  this. 

Five  emergency  procedures  and  six  checklists  were  selected  for  the  test.  By 
speaking  the  name  of  the  emergency  procedure  or  checklist  the  procedure  or 
checklist  is  selected  and  reported  back  by  voice  output  (see  Table  2).  Using  the 
voice  commands  "Okay",  "Affirm”,  "Last  item"  or  "Go  back”  the  pilots  could 
control  the  voice  output. 


Pilot 

System 

Main  generator  failure 

Main  generator  failure 

Okay 

Circuit  breakers  - 
check  in 

Okay 

Main  generator  reset  - 
then  on 

Table  2 


After  the  tests  pilots  rated  the  realizing  pf  voice  controlled  voice  output 
positively  for  this  application.  A  "ready  for  use"  installation  yet  must  allow  a 
more  flexible  handling  of  the  sequential  procedures,  e.g.  the  confirmation  of 
each  checklist  point  should  be  avoided.  Also  some  switches  are  located  close 
together  and  when  reading  a  written  checklist,  pilots  group  items  and  control 
the  corresponding  switches  all  together.  Such  an  adaptation  of  checklists  text 
and  procedure  is  required. 


2.3  Speech  filing  for  flight  plans,  weather  data  and  pilots'  notes 

At  present  pilots  use  a  writing  pad,  which  is  fixed  at  their  upper  thigh,  to  record 
via  radio  transmission  ordered  frequencies,  headings,  heights,  speeds,  air 
pressures  etc.  or  to  record  observed  targets  data  .  Handling  of  this  writing  pad 
may  result  in  critical  flight  situations.  Using  audio  tapes  to  record  and  replay  the 
notes  is  not  adequate  (mechanical  faults,  only  sequential  handling).  Therefore  a 
RAM-storage  device  for  voice  recording  was  tested.  Three  different  storage  areas 
for  altogether  three  minutes  of  speech  were  available  in  the  experiments. 
Recording  is  started  by  voice  command.  The  commands  "Speicher  Alpha",  "Spei- 
cher  8ravo"  and  "Speicher  Charlie"  select  the  corresponding  storage  and  the 
following  message  is  stored  until  the  pilot  releases  the  microphone  button.  By 
the  voice  commands  "Notiz  Alpha",  "Notiz  Bravo"  and  "Notiz  Charlie"  the  stor¬ 
ed  speech  is  replayed.  A  repeated  use  of  the  storing  command  deletes  the 
previously  stored  note. 

Handling  of  the  "voice  note  system"  has  been  shown  as  easy  but  pilots  need  time 
to  get  accustomed  with  it.  The  "voice  notebook"  has  been  used  for  storing  the 
always  necessary  repetition  of  radio  messages  or  short  keywords  as  a  reminder 
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for  some  later  examination.  Even  with  the  offered  three  different  storage  areas, 
the  sequential!  recall  of  stored  information  was  sometimes  boring  for  the  pilots, 
because  they  were  only  interested  in  the  stored  weather  data  but  not  in 
frequency,  flightplan  or  other  stored  data. 

A  more  “intelligent”  system  would  be  able  to  digitize  the  incoming  notes  and  to 
parse  them  in  accordance  to  certain  keywords  .  Weather  data  may  be  divided 
into  wind  heading,  -speed  etc.,  clearances  for  the  further  flight  pathes  could  be 
divided  similarly.  The  system  may  then  store  the  data  in  special  storage  areas, 
which  are  replayed  via  voice  output,  if  the  pilot  uses  the  corresponding 
commands  ("windheading”,  etc.). 


3.  Organizational  aspects  of  interactions  using  voice  input/output 

Besides  the  overall  positively  evaluated  use  of  voice  input  as  well  as  voice  output 
in  the  helicopter  simulator  necessary  improvements  for  reaching  a  stage  of 
practical  applicability  were  indicated. 

The  “problem  areas”  indicated  are: 

•  error  handling  in  voice  input 

•  flexibility  in  logical  structure  of  interaction 

•  adaptation  to  situational  context 


3.1  Error  handlinc 


Nowadays  voice  recognition  systems  evaluate  voice  utterances  without  any 
reference  to  previously  spoken  commands.  For  example,  the  substitution 
between  "Munchen  Tower"  and  "Minden  Tower"  could  be  avoided,  if  the 
system  were  connected  with  other  helicopter  components  and  therefore  knew 
the  helicopter's  actual  position.  With  additional  knowledge  about  frequency 
ranges  and  competences  the  system  would  be  aware  that  contacting  “Munchen 
Tower”  would  be  neither  possible  nor  useful  in  a  position  near  Minden.  A 
knowledge  base  about  pilots'  last  actions  as  well  as  available  flight  data  can 
improve  voice  input  to  speech  understanding.  As  a  first  step  a  flexible  syntax  and 
a  semantic  net  of  the  used  vocabulary  could  improve  voice  recognition  resultv 


3.2  Flexibilit 


The  coordination  of  simultaneous  tasks,  e.g.  with  both  hands,  is  highly  trained. 
The  voice  channel  for  radio  commands  is  treated  totally  independent.  When 
including  voice  i/o  as  normal  communication  channel,  on  the  one  hand  the  pilots 
must  be  aware  of  the  logical  structure  of  the  interactions  (priorities,  state- 
transactions)  and  on  the  other  hand  the  flexibility  must  not  be  reduced. 

In  the  experimental  system  an  additional  knob  located  on  the  control  stick  was 
used  for  activating  voice  control  to  separate  from  radio  communication.  By 
pressing  this  button  a  possibly  active  voice  output  was  stopped . 

This  selection  of  priorities  may  not  be  adequate  for  some  situations  where  a 
more  flexible  switching  between  emergency  procedures  and  radio  channels  is 
needed. 

For  more  flexible  structures  of  interaction  forms  a  rule  base  is  necessary  which 
can  decide  on  situational  context  which  priorities  are  adequate. 


-s- 


3.3  Situational  context 

A  received  radio  message  always  has  to  be  repeated  by  the  pilot.  Speech  filing 
(improved  by  word  parsing)  could  be  activated  automatically  when  the  pilot 
starts  the  repetition.  Another  example  where  situational  knowledge  can  be  used 
was  given  above  for  error  reduction  in  voice  recognition.  Even  for  handling 
emergency  procedures  knowledge  about  the  actual  flight  situation  can  be  used 
to  shorten  the  procedure,  speed  up  or  slow  down  voice  output  or  acceptable 
pilots'  reaction  times. 


4.  Conclusion 

Voice  i/o  has  been  shown  to  be  a  valuable  addition  to  man-machine 
communication  in  helicopter  applications.  For  development  of  a  "ready  to  use*- 
system  certain  problems  have  to  be  solved.  Some  ideas  have  been  presented  how 
more  "intelligent*  systems  including  knowledge  concerning  the  actual 
helicopter  situation,  helicopter  construction  data,  dialogue  history  and  special 
knowledge,  e.g.  about  frequencies  can  improve  overall  system  behaviour. 
Supported  by  this  knowledge  voice  i/o  could  be  integrated  in  helicopters  and 
would  be  a  contribution  for  pilots'  safety. 
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INTRODUCTION 


It  is  recognised  that  the  fighter  pilot  of  today  is  overloaded  in  the 
amount,  complexity  and  diversity  of  information  he  has  to  assimilate  and 
act  upon,  often  under  critical  time  constraints.  He  acts  as  an  integrator 
of  information  from  the  separate  aircraft  systems  to  build  mental  pictures 
of  the  state  of  his  aircraft  and  the  tactical  situation.  The  addition  of 
more  inputs  from  sensors,  tactical  communication  links  and  greater 
sophistication  of  veapons  and  other  systems,  threatens  to  overwhelm  the 
pilot. 

There  is  much  publicity  about  the  potential  advantages  offered  by  new 
cockpit  technologies  providing  'virtual'  or  'panoramic'  displays.  The 
suggestion  is  that  these  vill  improve  the  pilot's  situation  awareness,  or 
perhaps  more  accurately  his  perception  of  his  situation.  An  Improvement  in 
the  pilot's  perceptual  tasks  vill  not  necessarily  lead  to  an  improvement  in 
the  pilot's  cognitive  task  loading.  It  is  believed  that  the  form  of 
display  medium  is  of  only  minor  significance,  when  compared  to  the 
automation  strategies  adopted  in  the  design  of  the  avionic  system.  It  is 
possible  that  an  increase  in  the  sophistication  of  the  system  may  actually 
make  matters  vorse  for  the  pilot  in  terms  of  the  imposed  cognitive  load. 
This  is  why  the  approach  taken  must  properly  address  the  question  of 
automation  and  the  provision  of  a  flexible  allocation  of  function. 
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Furthermore,  to  suggest  that  improvements  in  the  pilot's  situation 
awareness  will  be  dependent  on  nev  display  technology,  is  to  ignore  the 
possibility  of  updating  not  only  the  aircraft  currently  in  service  but 
those  expected  in  the  next  5  -  10  years,  which  will  not  be  able  to  benefit 
from  such  technologies.  There  are  potentially  many  improvements  that  could 
be  made  without  completely  gutting  and  refitting-  the  cockpit. 

In  both  the  short  and  long  term  therefore,  effort  needs  to  be  directed 
tovards  avionic  system  design  driven  by  the  actual  needs  of  the  pilot.  The 
emphasis  should  be  on  ensuring  that  the  pilot  is  provided  with  the  right 
information,  in  the  right  form,  at  the  right  time,  and  that  the  right  tasks 
are  automated. 


APPROACH 


A  three  pronged  approach  is  required.  Ve  must  first  get  a  framework 
for  the  identification  of  pilot  needs.  Liaison  with  experienced  pilots 
should  provide  the  necessary  first  step.  At  this  stage  it  is  important 
that  discussions  centre  upon  specific  missions  and  specific  classes  of 
problems  experienced  by  aircrew,  so  that  effort  can  be  directed  towards 
specific  problem  areas.  It  would  be  all  too  easy  to  embark  upon  a  lengthy 
programme  of  work  to  automate  functions  for  which  there  are  obvious 
engineering  solutions  but  vhich  do  not  really  help  the  pilot. 

In  parallel  to  this  we  must  develop  a  metric  of  situation  awareness, 
so  that  prospective  Improvements  can  be  objectively  assessed.  It  vill  be 
noted  that  whilst  the  requirement  to  maintain  awareness  of  the  changing 
situation  has  become  a  major  design  objective  (Taylor,  1987)  a  universal 
definition  of  what  situation  assessment  is,  let  alone  a  validated  metric, 
does  not  yet  exist.  Nonetheless  progress  is  being  made,  and  within  the 
near  future  such  metrics  should  be  in  use  at  a  number  of  establishments. 
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But  perhaps  the  most  important  aspect  for  the  future,  is  the  emerging 
workstation  technology  being  provided  by  the  likes  of  SUN,  SYMBOLICS, 
SILICON  GRAPHICS  and  TEXAS  INSTRUMENTS.  The  impact  of  these  will  be 
considerable,  since  the  time  cycle  for  creating  prototypes  of  quite  complex 
systems  is  of  an  order  shorter  than  vith  the  previous  generation  of 
equipment.  Moreover  the  time  taken  to  implement  or  amend  control 
relationships  or  complex  display  suites  can  be  very  rapid  indeed.  As  a 
consequence  the  emphasis  can  move  away  from  some  of  the  more  esoteric 
modelling  activities  associated  vith  predicting  pilot  performance,  and 
concentrate  upon  a  more  pragmatic  approach  vhereby  the  efficacy  of  a  number 
of  potential  control  and  display  strategies  can  be  quickly  ascertained. 

Thus  the  approach  must  be  one  in  which  the  central  activity  is  geared 
to  the  generation  of  a  series  of  prototypes,  driven  by  a  set  of  goals 
derived  from  current  problems,  and  supported  by  evaluation  using  a  suitable 
metric  of  situation  awareness. 

PROBLEMS  WITH  AUTOMATION 

The  provision  of  a  flexible  automation  strategy  has  long  been 
recognised  as  an  important  goal  for  system  designers,  most  recently 
referred  to  by  Lind  (1988).  The  potential  benefits  are  obvious.  In  times 
of  high  stress  the  pilot  should  be  able  to  sit  back  and  let  the  system 
handle  the  majority  of  the  functions,  leaving  him  time  to  make  effective 
executive  decisions.  At  other  times  he  should  have  access  to  whatever 
level  of  control  he  feels  to  be  appropriate.  The  latter  is  important  if  a 
loss  of  pilot  skill  is  to  be  avoided  and  if  he  is  to  develop  a  thorough 
understanding  of  the  system  and  to  have  a  high  degree  of  confidence  in  it. 
Prototyping  vill  allow  the  exploration  of  such  strategies. 

Whilst  the  identification  of  current  pilot  problems  provides  an 
important  starting  point,  real  progress  can  only  come  from  pilot 
interaction  vith  a  system  in  a  real  time  environment.  In  this  respect 
prototyping  satisfies  a  number  of  objectives.  It  enables  one  to  test 
control  strategies  and  improve  them,  it  helps  to  ellicit  information  on 
less  obvious  pilot  skills  suitable  for  support  by  automation  and  it  helps 
to  identify  new  automation  requirements. 
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PROTOTYPE  DEVELOPMENT 

The  major  problem  facing  someone  hoping  to  develop  a  prototype  of  a 
modern  avionics  system  and  Che  environment  in  which  it  and  the  aircraft 
will  operate,  is  the  complexity  of  the  situation.  A  method  for  overcoming 
the  more  onerous  time  consuming  aspects  of  this  is  currently  under 
investigation.  Figure  1  summarizes  a  potential  configuration. 


Figure  1.  Proposed  Prototype  Configuration 
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Such  a  system  would  have  three  functional  areas;  on*  providing  a  basic 
simulation  of  a  mission  scenario  and  of  a  very  elementary  avionic  system, 
one  providing  a  reconfigurable  cockpit  and  the  third  providing  a  Cockpit 
Manager  Emulator.  The  concept  is  that  a  pilot  flies  a  mission  from  a 
simple  cockpit,  whilst  behind  the  scenes  an  operator  (or  possibly  a  team  of 
'experts')  manipulate  the  scenario  from  another  workstation.  A  series  of 
manipulation  rules  associated  with  each  class  of  parameter  being  changed, 
helps  the  operator  to  get  an  appreciation  of  hov  the  parameter  might  be 
interpreted  by  the  avionics  system  at  a  particular  rang*  or  in  a  particular 
jamming  environment. 

He  is  then  in  a  position  to  evaluate  the  potential  of  a  variety  of 
routes  or  other  actions,  based  upon  either  a  knowledge  of  what  is  actually 
happening  or  of  what  is  perceived  to  be  happening  by  the  avionic  systems. 
The  operator  then  judges  from  the  situation  the  importance  of  this 
particular  piece  of  information  and  decides  whether  the  pilot  should  be 
informed,  or  if  some  action  should  be  taken.  For  example,  it  could  be  of 
lov  priority  and  hence  for  'information  only'.  Alternatively,  he  could 
decide  that  a  particular  course  of  action  should  be  recommended  to  the 
pilot.  Finally  he  may  decide  that  the  time  available  is  such  that  the 
C'ckpit  manager  should  take  over  and  merely  advise  the  pilot  of  the  action 
taken. 

Alternatively,  the  operator  can  respond  either  to  demands  made  by  the 
pilot  of  his  cockpit  manager  or  by  observing  the  pilot's  current  situation, 
and  then  by  following  the  procedure  outlined  above  (ie  inform,  recommend  or 
implement). 

The  prototyping  activity  thus  concentrates  upon  identifying  the 
decisions  and  actions  to  be  made  by  the  pilot  and  upon  the  decisions  and 
actions  that  could  be  taken  by  the  avionic  system.  Vhilst  a  significant 
effort  will  be  required  to  provide  the  cockpit  facility,  this  could  no 
longer  be  regarded  as  the  core  of  the  system.  This  now  shifts  to  the 
development  of  the  Cockpit  Manager  Emulator,  and  rules  and  information 
ellicited  during  its  development. 


58 


Page  6 


CONCLUSION 

It  is  expected  that  this  type  of  prototyping  activity  vill  yield 
important  information  as  to  the  efficacy  of  a  variety  of  proposed 
automation  strategies  and  also  of  types  of  interfaces  that  will  be  required 
and  of  the  sort  of  skills  that  will  be  required  to  operate  such  systems. 
It  is  believed  that  more  vill  be  learnt  from  the  development  of  the  Cockpit 
Manager  Emulator  than  vill  emerge  from  developments  of  the  reconfigurable 
cockpit.  The  exercise  vill  be  an  important  stage  in  the  generation  of 
future  requirements  for  avionic  system  design. 

REFERENCES 


1.  R.M.  Taylor  (1987)  "Human  Factors  Implications  of  Navigation  in 
Advanced  Aircrev  Systems"  in  Proceedings  of  the  Royal  Institute  of 
Navigation  and  the  Nautical  Institute  Joint  Seminar  on  the  Human  Factor  in 
Navigation  held  at  the  Tavistock  Institute,  9  December  1987 

2.  J.H.  Lind  (1988)  "Cockpit  Displays  For  a  Knowledge-Based  System:  The 
Intelligent  Air  Attack  System"  in  NAECON  1988,  pp  918-925. 


59 


Validating  On-Line  Models  of  Activity  Patterns: 

Getting  Machines  to  Meat  Operator  Needs  Tor  5upport 

Walter  0.  Seward 
Gerald  P.  Chubb 
SofTech,  Inc. 

3100  Presidential  Drive 
Fairborn,  OH  45324  USA 

Summary 

Rapid  coordination  of  activities  requires  efficient  communication  of  Information, 
requiring  a  prediction  and  validation  of  assumed  versus  actual  Information  needs. 
The  problem  is  to  predict  what  will  be  needed  next  In  supporting  a  dynamically 
fluctuating  task  stream.  This  Is  a  scheduling  problem  not  only  faced  by  pilots 
but  by  designers  of  real-time  operating  systems  software.  Techniques  being 
developed  to  control  resources  in  multi -processor  distributed  avionics  systems 
appear  useful  for  modeling  pilot  task  management  strategies  and  decision  making. 
Proposed  strategies  for  resource  allocation  and  load  alleviation  are  presented, 
along  with  the  measures  used  to  evaluate  performance.  Information  analysis  and 
representation  methods  are  reviewed  as  ways  to  capture  task  demands  In  terms  of 
implied  resource  requirements.  Limitations  in  these  techniques  for  building  and 
validating  models  congruent  with  pilot's  perceptions  of  the  air  battle  situation 
are  addressed.  This  paper  also  reviews  how  metrics,  criteria  conflicts,  and 
performance  prediction  methods  are  used  In  computer  science,  proposing  suitable 
analogs  that  might  be  used  to  monitor,  measure,  and  predict  pilot  performance 
on-line.  Development  of  such  modeling  and  validation  methods  Is  essential  for 
expert  systems  to  anticipate  and  adapt  Intelligently  to  changing  pilot  needs  for 
Information. 

I.  INTRODUCTION 

Artificial  Intelligence  (AI)  embedded  In  an  expert  system  (ES)  can  provide  a 
pilot's  associate  or  electronics  crewmember  (EC)  of  varying  utility.  To  be  most 
useful,  the  EC  must  anticipate  the  pilots  need  for  help,  presenting  only  what  is 
useful  and  only  when  needed.  That  requires  predictive  anticipation  of  pilot 
actions.  To  anticipate  what  comes  next,  the  EC  will  need  to  model  and  predict 
the  pilot's  behavior.  To  predict  accurately,  the  EC  must  validate  Its  model  of 
the  pilot  on-line  and  adapt  accordingly. 

1.1  The  Validation  Problem 

The  psychological  theory  of  tests  and  measurements  proposes  three  basic  kinds  of 
validity:  context,  construct,  and  criterion.  Context  validity  deals  with 

whether  sample  test  items  cover  the  phenomenon  under  test.  Construct  validity 
checks  for  consistency  between  the  phenomenon  tested  and  the  character  of  the 
test  Items.  Criterion-related  validity  compares  test-based  predictions  against 
observed,  measured  behavior.  Criterion  validity  suffers  if  measures  are  not 
reliable  or  If  construct  and  context  validity  are  weak.  In  all  cases,  the 
measure  used  to  quantify  the  degree  of  validity  Is  the  Pearson  product  moment 
correlation  coefficient,  or  In  some  cases,  its  non-pa rametrlc  counterparts  (e.g. , 
Spearman  Rho). 

These  methods  fall  short  on  two  counts:  (1)  they  are  global  measures,  and  (2) 
they  require  ordinal  or  Interval  measures  of  all  variables.  Validation  of 
operator  activities  In  an  on-line,  dynamic  environment  requires  a  new  approach: 
one  more  sensitive  to  discrete  variables  and  nearly  continuous  monitoring  of 
selected  variables  to  detect  almost  instantaneously  certain  critical  changes  in 
state  (of  the  environment,  the  system,  and  the  operator).  Moreover,  these 
validation  methods  need  to  be  robust  and  insensitive  to  certain  kinds  of  missing 
data.  Not  everything  the  pilot  does  will  be  measurable.  Models  of  cognitive 
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processes  are  only  Indirectly  testable  In  terms  of  observable  behavioral 
consequences,  since  the  processes  themselves  are  Inherently  non-observable 
directly. 

1.2  An  Analogy:  Multitask  Operating  Systems  (MTOS)  and  Distributed  Systems 

Software  for  computers  embedded  In  advanced  aircraft  avionics  systems  have  an 
executive.  The  executive  software  that  controls  these  airborne  computers  must 
handle  the  scheduling,  resource  allocation,  and  management  functions  that  service 
unpredictable  Inputs  and  produce  required  outputs  within  specified  time  frames. 
Many  of  the  performance  requirements  Imposed  on  real-time  executives  or  operating 
systems  In  distributed,  multi-task  processing  environments  are  similar  to  the 
requirements  pilots  face  as  cockpit  managers.  (Chubb,  et  al,  1987.)  Moreover,  AI 
concepts  (such  as  daemons)  were  first  Implemented  In  operating  system  software. 

Real-time  computing  systems  are  systems  which  must  interact  with  their 
environment  under  precise  timing  constraints.  The  timing  constraints  require 
that  the  system  recognize  an  external  event,  perform  required  computational 
tasks,  and  emit  data  or  control  signals  within  sufficient  time  to  affect  the 
environment.  The  tasks  which  characterize  embedded  real-time  computer  systems 
applications  can  be  categorized  as  follows:  (1)  hard  real-time  tasks  which  must 
complete  each  processing  request  within  a  specified  deadline  or  a  catastrophic 
system  failure  will  occur;  (2)  soft  real-time  tasks  which  do  not  cause  a 
catastrophic  system  failure  If  their  deadlines  are  not  met,  but  the  value  of  the 
results  decrease  as  a  function  of  the  time  after  the  deadline;  and  (3)  tasks 
which  are  not  time-critical  and  therefore  do  not  have  an  associated  deadline  for 
completion. 

A  hard  real-time  system  is  a  system  that  contains  any  hard  real-time  task. 
Examples  of  hard  real-time  systems  Include  digital  avionics.  Industrial  process 
control,  coumand  and  control,  and  flight  control  systems.  The  system  executive 
Is  responsible  for  allocating  available  system  resources  (sensors,  processors, 
actuators)  such  that  all  hard  real-time  tasks  meet  their  deadlines;  the 
degradation  of  overall  system  Is  minimized  by  failure  of  soft  real-time  tasks  to 
meet  their  deadlines;  and  maximize  the  value  of  the  non  time-critical  tasks 
completed.  The  executive  must  also  satisfy  all  sequencing  requirements  of  the 
task  set  and  develop  a  schedule  for  each  resource  even  when  not  all  tasks  are 
known  a  priori.  Typically,  the  resources  available  are  very  scarce;  satisfactory 
allocation  solutions  are  not  easily  attainable;  and  reallocation  must  frequently 
consider  concurrent  actions  by  many  of  the  system  elements  to  complete  the 
required  processing. 

1.3  Relevance:  MTOS  Behavior  Analysis  Methods 

Task  scheduling  has  been  studied  extensively  for  both  computer  and  operations 
research  applications.  (Casavant  and  Kuhl,  1988;  Cheng,  et  al,  1987;  French, 

1986. )  Ti  basic  concepts  of  resource  allocation  transcend  the  specific 
application.  However,  the  specifics  of  task  scheduling  environments  dictate  the 
particular  techniques  which  have  the  most  direct  application.  Comprehensive 
summaries  of  different  facets  of  task  scheduling  as  related  to  computer  systems 
Include  distributed  processor  scheduling  of  hard  real-time  tasks.  (Cheng,  et  al , 

1987. ) 

The  allocation  of  resources  In  most  application  environments  is  a  computationally 
Intensive  process.  Many  such  problems  are  In  fact  NP-complete.  (Papadimltrlou 
and  Stelglitz,  1982.)  The  more  complex  the  system  (In  terms  of  numbers  of 
resources  to  be  allocated),  Interdependence  of  tasks,  and  operational  performance 
constraints,  the  more  difficult  the  scheduling  problem.  Invariably  there  arises 
a  conflict  In  terms  of  resource  availability  and  the  stated  system  performance 
metrics,  which  further  Increases  the  complexity  of  the  problem.  Furthermore,  the 
performance  metrics  used  often  represent  conflicting  goals  and  their  relative 
Importance  is  application  dependent. 
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The  performance  of  a  task  scheduling  algorithm  for  hard  real-time  systems  must  be 
evaluated  not  only  In  terms  of  satisfaction  of  precedence  and  timing  constraints, 
but  also  in  terms  of  the  degree  to  which  the  schedule  enhances  the 
fault-tolerance  of  the  system's  architecture  and  the  use  of  the  system's 
resources.  Tor  example,  an  algorithm  which  tends  to  schedule  successor  tasks  on 
processors  different  from  the  predecessor  task  may  needlessly  increase  the  load 
on  the  shared  system  coomuni cation  mechanism  and  at  the  same  time  decrease  the 
system's  reliability  by  adding  another  potential  failure  point  for  that  task  set. 
The  conflict  between  the  distribution  of  tasks  among  the  processors  in  order  to 
achieve  a  balanced  load  at  the  expense  of  Increased  communications  load  must  also 
be  resolved  In  light  of  the  system's  reliability. 

1.4  Limitations  of  the  Analogy  and  Reasonable  Expectations 

The  design  and  Implementation  of  verifiable  scheduling  algorithms  for  real-time 
distributed  systems  Is  a  topic  of  extensive  on-going  research  which  has  yet  to  be 
satisfactorily  solved.  Yet  successful  development  of  an  electronic  crewmember 
depends  upon  validation  of  the  pilot  model  and  must  be  addressed.  Avionics 
software  also  requires  verification  and  validation  prior  to  deloyment. 

Because  there  seems  to  be  a  close  correspondence  between  human  cognitive  proces¬ 
ses  and  MTOS  design,  the  procedures  and  techniques  used  to  verify  and  validate 
such  software  may  provide  Insight  on  possible  techniques  for  on-line  validation 
of  pilot  models.  The  concept  is  to  model  human  cognition  as  if  It  were  an  MTOS 
and  then  examine  how  that  model  might  be  validated  from  on-line  monitoring  of 
behavior. 

2.  THE  CHARACTER  OF  THE  COGNITIVE  MOOEL  VALIDATION  PROBLEM 

Imagine  a  computer  like  the  Digital  Equipment  Corporation  (DEC)  model  PDP-11. 
This  computer  could  be  used  many  ways.  It  has  been  used  in  a  variety  of 
real-time  applications. 

There  were  two  commonly  available  operating  systems  for  this  machine:  RT-11  and 
RSX-11.  Either  could  control  the  machine  and  handle  real-time  processing. 
However,  Internally  they  were  designed  differently  and  therefore  behaved  differ¬ 
ently  Inside  the  PDP-11.  However,  casual  observation  of  the  external  behavior  of 
the  PDP-11  would  rarely  reveal  whether  RT-11  or  RSX-11  was  In  control. 

The  analogy  to  the  pilot  Is  quite  simple.  Each  pilot  comes  with  the  same 
anatomical  design,  but  each  Individual  has  similar  but  different  cognitive 
processing  behaviors  (one's  own  MTOS)  that  have  been  learned.  From  observed 
behavior,  one  can  sometimes  Infer  that  there  must  be  a  difference  between 
cognitive  processing  used  by  one  individual  versus  another,  but  reasons  for  the 
differences  cannot  be  experimentally  confirmed.  One  cannot  directly  manipulate 
the  cognitive  processes  but  only  the  Inputs  to  those  processes. 

Moreover,  It  is  not  possible  to  ascertain  from  simple  stimulus  response  analysis 
the  situation  perceived  by  the  human  operator  at  a  given  instance  of  time;  and 
yet,  this  assessmenFTf  the  situation  is  a  critical  element  of  an  EC  and  the 
associated  pilot  model,  as  will  be  discussed  in  section  3. 

Consequently,  studying  cognitive  behavior  from  the  perspective  of  stimulus 
response  psychology  will  not  uncover  the  underlying  cognitive  processes  anymore 
than  studying  computer  inputs  and  outputs  will  reveal  MTOS  design.  Also,  perfect 
knowledge  of  neuroanatomy  Is  no  more  useful  than  knowing  the  hardware  design  of 
the  PDP-11.  Such  circuit  Information  cannot  reveal  th;.*  design  differences 
between  RT-11  and  RSX-11. 

On  the  other  hand,  because  we  know  the  design  differences  between  RT-11  and 
RSX-11,  It  Is  possible  to  analyze  their  behavior  differences  and  postulate  where 
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their  respective  behaviors  will  be  distinctly  different  versus  Indistinguishable. 
By  analogy,  postulating  various  MTOS  designs  as  hypothetical  constructs  of 
cognitive  processing,  one  has  a  basis  for  postulating  testable  differences  In 
human  behavior. 

By  progressive  refinement  and  testing,  one  may  then  find  a  suitable  set  of  MTOS 
models  which  behave  equivalent  to  human(s).  That  does  not  mean  humans  In  fact 
operate  as  the  MTOS  suggests.  It  only  means  the  EC  can  use  a  model  that  Is 
equivalent  In  the  context  of  Interest:  predicting  what  pilots  may  do  next.  It 
Is  unlikely  that  one  model  will  fit  all  situations.  For  example,  in  some  situa¬ 
tions  the  Individual  might  behave  more  like  RT-11  and  In  other  cases  more  like 
RSX-11.  Then  the  EC  needs  on-line  tests  that  will  discern  the  situation  and 
switch  to  the  most  appropriate  model. 

3.  CUSSES  OF  MODELS:  LAYERING 

Actually,  three  classes  of  dynamic  models  are  required  and  they  assume  a  fourth 
static  model.  First,  there  has  to  be  a  model  of  the  environment.  This  consti¬ 
tutes  a  situation  awareness  model.  Second,  there  needs  to  be  a  set  of  procedural 
models  describing  executable  activities.  Third,  there  needs  to  be  the  MTOS 
model.  This  model  buffers  the  events  generated  by  the  actual  environment, 
updates  the  situation  awareness  model,  and  on  the  basis  of  reasonable  Inferences 
alters  the  scheduling  and  Implementation  of  executable  activities.  The  fourth 
model  Is  the  Intent  structure  or  knowledge  base  that  is  drawn  upon  by  MTOS  to 
bridge  the  gap  between  situational  awareness  and  the  selection  of  appropriate 
procedures. 

From  these,  one  can  generate  a  fifth  model:  the  data  describing  actual  real-time 
behavior  of  the  three  dynamic  models,  conditional  on  the  static  intent  structure. 
(Clearly,  the  Intent  structure  may  be  updated  and  modified  based  on  combat 
experience,  but  In  any  one  scenario,  the  total  structure  should  remain  static 
since  It  defines  a  set  of  goals,  their  priorities,  and  rules  for  resolving 
conflicts).  This  fifth  model  Is  the  basis  for  validating  the  MTOS  model:  it  is 
the  documented  set  of  observations  of  man-machine  behavior  (not  all  aspects  of 
behavior  are  totally  capturable,  hence  observations  only  model  behavior). 

4.  VALIDATION  ANALYSIS 

In  IV&V,  testing  requirements  are  defined  top-down,  and  Implemented  bottom-up. 
The  MTOS  design  can  be  described  in  a  top-down  fashion  using  a  Structured  Analy¬ 
sis  and  Design  Technique  (SAOT®).  At  the  lowest  level,  MTOS  eventually  passes 
control  to  other  processes:  these  are  the  procedural  models  that  execute  in 
response  to  environmental  events  (stimuli)  subject  to  MTOS  control  (41a  the 
defined  Intent  structure). 

The  activity  switch  points  occur  in  time,  represent  a  state  transition  (from  A  to 
B),  and  require  a  particular  resource  (eyes,  hand,  voice,  etc.).  To  be  valid, 
the  MTOS  must  generate  similar  patterns  of  state  transitions  In  comparable  time 
periods.  Comparability  Is  the  issue.  A  distinction  must  be  made  between  syn¬ 
chronous/asynchronous  and  hard  versus  soft  timing  requirements. 

Synchronous  events  are  those  where  two  or  more  events  should  occur  at  the  same 
time.  That  timing  requirement  may  be  either  hard  or  soft.  Clearly,  scheduling 
hard  synchronous  events  In  a  dynamic  real-time  environment  is  challenging.  But 
these  are  key  points  where  the  MTOS  must  match  pilot  behavior  to  be  a  valid 
model.  For  soft  or  asynchronous  events,  greater  variation  in  behavior  can  be 
allowed  without  invalidating  the  MTOS  model  as  suitably  predictive. 
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Since  the  MTOS  lives  between  two  models  (procedural  and  situational)  while 
depending  on  a  third  (intentional),  its  validation  rests  on  first  validating  the 
other  three  models.  If  the  MTOS  model's  behavior  fails  to  correspond  to  observed 
pilot  actions,  one  can  either  infer  the  MTOS  can  be  Improved  or  one  of  the  other 
models  mav  also  be  Invalid.  Within  the  MTOS,  there  are  two  major  sources  for 
error:  1)  the  nature  and  relationships  among  Incorporated  processes,  or  2)  the 
control  structure  that  governs  the  switching  among  processes  internal  to  the 
MTOS.  Again,  the  data  will  not  be  diagnostic,  but  experimentation  with  the  MTOS 
architecture  can  produce  variations  which  may  better  match  observed  behavior 
patterns. 

4.2  Other  Considerations 

Links  also  need  to  be  made  to  flight  test  efforts:  research  and  development  test 
and  evaluation  (ROTJE),  its  operational  equivalent  (OTSE),  and  subsequent  spe¬ 
cialized  test  programs.  The  only  way  to  Integrate  these  results  though  is  to 
have  developed  a  descriptive  model  of  the  decomposition  of  performance  require¬ 
ments  into  the  set  of  testable  behaviors  that  are  measurable  In  the  airborne 
versus  ground  environments. 

On-line  validation  of  the  pilot  model  requires  continual  comparison  of  predicted 
and  actual  pilot  behavior.  The  EC  must  anticipate  pilot  actions  based  upon  its 
assessment  of  the  environment  (which  may  differ  significantly  from  the  pilots' 
perceptions),  the  Intent  Structure,  and  the  Procedural  Models.  In  addition,  the 
model  must  account  for  all  possible  actions  which  might  be  taken  by  a  pilot  from 
a  given  state,  and  respond  within  a  time  frame  which  is  compatible  with 
"real-time*  requirements.  Furthermore,  the  EC  must  be  able  to  respond 
appropriately  to  predict  pilot  behavior  when  degraded  performance  occurs  due  to 
injury  or  the  effects  of  a  chemical,  biological,  or  nuclear  environment. 
Adequate  test  and  model  development  are  not  possible  to  handle  all  contingencies. 
The  spectrum  of  possible  responses  is  too  large  to  test  adequately  such  that 
system  failure  is  precluded.  The  scope  of  such  test  requirements  exceeds  even 
those  of  a  complex  software  system. 

Existing  techniques  appear  to  offer  little  hope  of  validating  on-line  models. 
They  have  not  been  successfully  applied  to  the  much  less  complex  environment  of 
MTOS  and  distributed  computing.  However,  an  alternative  to  a  completely 
validated  system  Is  a  system  which  can  rapidly  adapt  to  a  "new"  unknown  stimulus 
response  set.  Maturation  of  neural -network  technology  may  provide  the  required 
capabilities  to  make  the  electronic  crewmember  a  reality. 
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INTRODUCTION 

A  dacision  undar  uncartainty  can  ba  defined  as  a  decision  whara  thara  is  more  than  one  decision  option 
with  a  probability  of  success  of  greater  than  zero,  and  the  probabilities  and  utilities  of  the  outcomes  can 
not,  a  priori,  be  definitely  known.  Thus  a  decision  support  system  can  ba  defined  as  an  external  means 
of  reducing  the  uncertainty  associated  with  a  given  decision,  thus  facilitating  the  decision  maker's 
behaviour.  Since  probabilistic  Judgement  is  implicit  In  any  decision  under  uncertainty,  this  paper  sets 
out  to  explore  the  use  of  oven,  system-generated  probability  estimates  as  an  interface  methodology 
for  future  decision  support  system  design.  An  attempt  is  made  to  develop,  from  initial  experimental 
work,  an  understanding  of  the  Information  Processing  involved  in  the  use  of  such  information  and  to 
tentatively  examine  the  implications  of  such  processes  for  the  Electronic  Crewmem  concept. 

Many  potential  advantages  and  applications  of  Intelligent  Decision  Support  in  the  mtiuary  environment 
have  been  documented  [  1,  2]. So  mo  of  the  particular  implications  for  the  single-seat  high  performance 
aircraft  cockpit  are  described  by  [  3]  .  They  discuss  the  use  of  uncertain  data  by  the  EC  and  describe 
two  possible  approaches  by  the  EC  to  using  that  data.  Firstly,  the  EC  could  represent  the  uncertainty  to 
the  pilot  e.g.  by  probability  tags'  thus  allowing  the  pilot  to  resolve  the  uncertainty,  or  secondly  the  EC 
could  resolve  me  uncartainty,  using  for  example  Rules  Of  Engagement,  thus  reducing  the  decision 
workload  on  the  pilot  The  danger  in  removing  the  pilot  from  the  decision  process  is  that  awareness  of 
the  situation  can  ba  lost  resulting  in  inappropriate  behaviour  later  in  the  mission.  For  this  reason,  the 
present  study  has  focussed  on  attempting  to  ascertain  the  usefulness  of  providing  enhanced,  explicit 
probability  information  to  the  pilot  in  the  form  of  probability  labels  for  each  decision  option.  The  aim  of 
the  research  is  to  attempt  to  clarify  the  extent  to  which  the  provision  of  such  information  can  produce  a 
performance  benefit  for  the  decision  maker,  and  to  by  to  spot  any  potential  disadvantages  with  its  use. 

The  use,  and  misuse,  of  probabilistic  information  by  its  Heuristical  rather  than  Statistical 
application  by  human  decision  makers  is  well  documented  (see  [  4]  for  an  overview).  It  has  been 
suggested  that  people  do  not  use  statistical  probability  estimates  rationally,  and  that  Rule-Of-Thumb 
interpretations  can  lead  to  fallacious  judgements.  The  external-validity  of  these  findings  have  been 
queried,  however,  by  [  S],  who  suggest  that  real-world  heuristic  decisions  are  more  rational  than 
laboratory  studies  might  Indicate.  The  types  of  decisions  used,  whilst  being  easy  to  control 
experimentally,  have  tended  to  be  either  simplistic;  unrealistic;  abstract;  or  unfamiliar.  To  this  end. 
the  present  study  used  'rear  motoring  navigation  decisions  in  a  pseudo-dynamic  context  to  try  to 
preserve  both  external  and  ecological  validity  sufficiently  to  be  able  'to  generalise  any  results  to  the 
applied  aviation  context.  Thus  the  tasks  were  designed  to  map  onto  subjects'  existing  knowledge 
structures  so  as  to  enable  a  meaningful  judgemental  decision  to  be  made  on  them.  The  need  to  inculcate 
trust  in  the  system-generated  probabilities  was  considered  important  to  facilitate  their  use  [  •].  For 
this  reason  subjects  were  instructed  that  the  computer  would  not  deliberately  'lie*  to  them  and  would 
always  generate  its  *  beet  guess*  or  nothing  at  ail.  This  was  reinforced  by  probability  labels  always 
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being  allocated  to  their  eonoct  option. 

It  mu  hoped  to  demonstrate  by  tMo  Initial  experiment  that  tha  proviaion  of  system  generated 
probability  labaia  to  daciaion  makers  will  haip  thorn  to  maka  thair  daclaion  mora  quickly  wharo 
uncertainty  la  raducad,  without  proving  to  hava  penalties  on  memory  lor  the  dadsiona  themselves.  Tha 
tactical  importance  of  maintaining  awaranaai  of  tha  daciaion  anvlronmanta  la  likoly  to  ba  important  and 
thus  mamory  for  previous  dacisJono  Mrffl  ba  nocsssary  to  preserve  tha  'Big  Picture'.  Thus  a  trade  off 
between  Response  Tima  (RT)  and  raievant  memory  details  would  potentially  ba  of  little  use  to  the 
operator. 


SCENARIOS 

An  example  of  the  acenarioe  used  is  given  below.  Each  one  comprised  a  problem  space  containing  the 
demand  criteria  which  the  decision  had  to  meet  This  was  presented  aa  a  paragraph  of  text  which  was 
presented  before  the  decision  options.  This  was  not  time  dependant  and  the  subjects  called  up  the  options 
whan  ready.  Three  options  were  given  for  each  scenario  and  were  presented  with  or  without  probability 
labels.  Each  option  contained  three  piecee  of  information  :  a  name/identifying  label  not  contingent  to  tha 
decision;  and  two  pieces  of  related  Information  contingent  to  the  decision  criteria.  For  each  scenario  the 
options  were  designed  such  that  there  was  one  option  which  fully  satisfied  the  criteria  (Probable  Opt); 
one  which  contained  Insufficient  information  to  meet  tha  criteria  but  did  not  contradict  them  (Possible 
Opt);  and  one  which  dearly  contradicted  the  criteria  (Impossible  Opt).  Tha  highest  probability  estimate 
was  always  given  to  the  Probable  option  and  the  lowest  to  the  Impossible  option. 

EXAMPLE  SCENARIO  (text  in  parenthesis  was  not  displayed) 


You  approach  a  roundabout  at  the  edge  of  town.  The  roundabout  has  three  exits  signposted  A31, 
A315,  B315S  respectively.  You  cannot  remember  which  road  to  take,  but  remember  that,  on  your 
previous  visit  last  winter,  that  there  was  a  tree  on  the  comer  of  the  correct  road  which  was 
completely  bare  of  leaves.  You  examine  the  turnoffs  bearing  in  mind  that  any  trees  may  have  been 
cut  down  since  your  last  visit  (Decision  Problem  Space) 


OPTIONS 

(1)  A31  -  Newly  panted  sapling  -  p  »  25% 

(2)  A315  -  Mature  Pine  tree  •  p«  0% 

(3)  B3155  -  Mature  Oak  tree  -  p  -  75% 


(PoesWe  opt) 
(Impoestote  Opt) 
(Probable  Opt) 


METHOD 

The  variables  investigated  in  this  study  were; 

(a)  Tha  presenca/absance  of  probability  information  l.e.  whether  probability  labels  were  attached 
to  the  decision  options  to  attempt  to  reduce  the  uncertainty  experienced  by  the  decision  maker. 

(b)  the  clarity/ambiguity  of  such  information  i.e.  whether  the  difference  between  the  probabilty 
labels  dearly  Indicated  one  option  as  being  preferable  e.g.  75/25/0  %;  or  were  unclear  e.g. 
35/33/32  %  where  the  differences  between  options  were  smalt,  so  as  not  to  dearly  Indicate  one 
option  as  being  correct 

External  variables  were  controlled  for  by  matching  (as  far  as  possible  without  interfering  with 
meaning)  the  information  content  and  length  of  scenarios  and  decision  options.  Any  rsmaining 
variation  waa  balanced  by  a  Latin  Square  design. 

Three  performance  measures  were  taken  in  the  experiment  Reaction/Decision  Times  (RTs) 
were  taken  to  attempt  to  identify  the  amount  of  processing  occurring  under  the  different 
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experimental  condition*.  A  memory  test  (given  5  minuta*  after  finishing  tha  scenarios)  was  used  to 
attempt  to  gauge  tha  amount  of  information  which  had  baan  stored  in  Long  Term  Memory  (LTM) 
under  each  condition.  Rehearsal  was  prevented  during  the  daisy  by  the  us*  of  a  numerical  distractor 
task.  Since  memory  encoding  i*  dependant  on  the  amount  of  processing  each  piece  of  Information 
receives,  this  second  measure  can  also  be  taken  as  an  Indicator  of  the  Information  Processing 
demand  change*  induced  by  the  experimental  variables.  A  Confidence  rating  was  also  taken  to 
examine  the  subjects'  subjective  responses  to  the  probability  labels. 

BEautia 

Table  1  (below)  show*  a  summary  of  the  results  obtained  from  this  study.  The  totals  summed 
across  the  twelve  subjects  are  shown,  together  with  their  means  in  parenthesis.  It  can  be  seen  that 
response  times  were  significantly  faster  (p<0.01)  when  clear  Probability  information  (PI)  was 
given  than  in  either  of  the  other  two  conditions,  with  the  ambiguous  PI  producing  clearly  the 
slowest  times.  This  implies  that  the  processing  required  from  subjects  is  reduced  by  providing  the 
additional  source  of  Information  and  thus  removing,  part  at  least,  of  their  uncertainty.  The 
confidence  values  attributed  to  the  Clear  Pi  condition  were  significantly  higher  (p<O.Ol)  than  for 
the  other  two  group*.  Thus  the  provision  of  tit*  extra  information  appeared  to  make  subjects  more 
confident  in  their  decision*.  Again,  this  is  Ifcaiy  to  be  by  a  reduction  in  the  uncertainty  inherent  in 
the  decision.  This  interpretation  is  substantiated  by  a  correlation  of  0.938  (p<0.001)  between  the 
RT  and  Confidence  values,  thus  implying  the  same  source  of  variance  is  likely  to  be  causing  both 
effects.  The  memory  scores  show  a  main  effect  of  PI  type  (pcO.OOS)  with  scores  being  lowest  for 
the  dear  PI  condition.  It  can  be  seen  from  the  scores  for  each  option,  however,  that  the  majority  of 
this  variance  is  accounted  for  by  the  reduced  Impossible  option  scores.  This  implies  that  any 
reduction  in  the  processing  of  the  information  is  occurring  within  this  option.  When  read  in 
conjunction  with  tha  RT  values,  this  can  be  taken  to  indicate  that  reduced  processing  was  necessary 
to  reject  the  Impossible  option. 


CLEAR 

PROB.INFO. 

AMBIGUOUS 

PROB.INFO 

NO 

PROB.INFO 

RESPONSE 
TIMES  <s) 

12.47(1.04) 

16.01  (1.33) 

14.92(1.24) 

CONFIDENCE 

RATINGS 

13.54  (f  .30) 

13.29(1.11) 

13.63(1.14) 

a 

Of 

PROBABLE 

OPTION 

76(3.17) 

79(3.29) 

80  (3.33) 

$ 

S 

POSSIBLE 

OPTION 

60  (2.30) 

60  (2.30) 

73  (3.04) 

2 

2 

IMPOSSIBLE 

OPTION 

36  (1 .30) 

60(2.30) 

60  (2.50) 

TABLE  1  ■  showing  the  total  scores  across  subject*  (mean  scores  are  In  parenthesis)  on  each 
measure  for  each  probability  condition.  Memory  scores  are  shown  for  each  option  category. 
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otafiuaaiQM 

Thu*  it  can  be  Man  from  th*M  moults  that  a  performance  gain  Is  achieved  by  ths  provision  of 
unambiguous  probabilistic  information  to  tha  dacision  maker,  without  any  datrimant  to  his  memory 
for  tha  relevant  dacision  options.  This  gain  appear!  to  be  tha  result  of  ths  reduced  uncertainty 
associated  with  tha  dacision  whan  tha  evtra  source  of  information  is  given,  as  implied  by  both  tha 
faster  rsiponis  times  and  tha  increased  levels  of  confidence  reported  in  tha  decisions  The  lower 
memory  scores  for  the  Impossible  option  whan  Clear  PI  Is  given  imply  that  a  saving  is  made  on  the 
amount  of  information  naading  to  be  processed  before  a  dacision  is  made  by  facilitating  tha 
rejection  of  this  option.  No  significant  memory  interference  Is  found  with  tha  other  options,  thus 
implying  that  processing  of  them  Is  not  largely  affected.  There  la  an  ncreasa  in  the  response  times 
whan  tha  PI  is  ambiguous,  as  compared  to  tha  No  PI  condition,  but  no  significant  difference  in  the 
memory  scores.  This  would  imply  that  an  Initial  attempt  is  being  made  to  um  the  PI,  but  when  this 
proves  unsuccssful  (because  of  the  unclear  nature  of  the  information)  then  processing  is  carried  out 
as  if  no  PI  were  available.  This  pattern  of  results  would  tend  to  suggest  a  Hypothesis 
acceptance/re jeetion  modal  of  the  decision  process,  whereby  the  probability  labels  are  being  used 
to  generate  an  initial  hypothesis  as  to  which  option  Is  correct,  perhaps  obviating  the  need  to 
generate  a  subjective  probability  estimate.  Tha  options  are  then  checked  against  this,  with 
subjective  expected  utilities  (or  equivalent)  being  calculated,  until  a  criterion  for  acceptance  or 
rejection  for  each  option  ia  reached.  Where  the  PI  is  clear,  then  RTs  are  faster  by  providing  a 
hypothesis  which  primes  the  zero  probabifty  option  to  be  rejected.  Where  the  PI  is  unclear,  then  no 
meaningful  hypothesis  can  be  generated  and  the  PI  information  is  discarded.  Describing  this  process 
in  terms  of  Neural  Networks  [  7],  it  could  be  argued  that  tha  Clear  PI  will  excite  nodes  on  the  two 
pathways  corresponding  to  tha  Possible  and  Probable  options  whilst  inhibiting  the  pathway  for  the 
Impossible  option.  Thus  the  response  options  available  will  be  effectively  reduced  to  two,  thus 
facilitating  tha  choice  of  the  correct  option.  Where  ths  PI  is  unclear,  each  pathway  will  be  excited 
almost  equally  by  the  labels,  thus  providing  little  or  no  assistance  to  tha  decision  maker.  Such  a 
model  can  only  be  advanced  tentatively  from  this  single  set  of  results,  but  may  provide  a 
framework  for  future  research. 

Thus  although  this  experiment  only  goes  a  very  small  way  towards  describing  the  processes 

involved  In  decision  making  under  uncertainty  and  the  effect  of  the  explicit  PI  lags  on  those 

deciaiom,  some  conclusions  can  nonathefesa  be  drawn.  Tha  possible  advantages  to  be  accrued  from 
tha  um  of  these  labels  In  ths  Pllot/EC  context  are  twofold,  when  theM  estimates  clearly  separate 
between  option!,  with  both  speed  of  response  and  confidence  Increasing.  A  third  benefit  could  be 
said  to  be  tha  reduction  in  the  memory  for  Irrelevant  options,  thus  reducing  the  memory  demands 
on  tha  pilot.  There  are,  howevsr,  many  questions  still  to  ba  answered  before  the  um  of  PI  tags 
can  be  recommended  as  a  design  feature  of  Decision  Support  systems.  How  large  must  the 
differences  between  the  PI  estimates  be  for  them  to  be  effective?  How  often  will  the  real-world 
aviation  context  allow  theM  clear  distinctions  to  be  made  available?  What  is  the  best  method  of 
representing  theM  probabilities  e.g.  digital  vs.  graphical/analog?  How  doss  trust  in  the  system 
affect  the  interpretation  of  such  labels?  Will  Incorrect  labels  lead  to  pilots  making  fallacious 

judgements  which  might  otherwiM  have  been  correct?  TheM  are  just  a  few  of  the  quMtiona 

towards  which  research  should  be  directed.  It  does  seam  clear  however  that  the  potential  benefits 
from  the  reduction  of  uncertainty  without  the  Iom  of  pilot  mandate,  provided  by  this  approach. 
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justify  the  sffort  involved  in  such  future  research. 

A  final  consideration  is  the  applicability  of  taking  an  Information  Processing  approach  to  Decision 
Support  system  design.  It  could  be  argued  that  it  is  knowledge  rather  than  information  which  is  the 
crucial  element  in  decision  making  under  uncertainty,  and  as  such  it  is  the  application  of  knowledge, 
not  the  way  information  is  processed,  that  is  the  key  to  understanding  (and  ultimately  recreating 
artificially)  the  intelligent  decision  process.  From  the  applied  viewpoint,  however,  the  ability  to 
enhance  understanding  and  awareness  by  the  correct  moding  of  information  available  to  the  decision 
maker  may,  in  the  short  term  at  least,  provide  a  greater  benefit  in  the  design  of  either  intelligent 
decision  support  or  a  fully  fledged  Electronic  Pitot  Associate.  It  is  for  this  reason  that  the  present 
study  has  put  epistemological  considerations  aside  in  favour  of  trying  to  ascertain  the  information 
requirements  of  the  decision  maker  in  terms  of  the  way  such  information  is  amalgamated  into  the 
decision  process.  Whist  the  pilot  is  still  in  the  cockpit  it  seems  sensible  to  gear  the  EC  to  support, 
rather  than  to  replace,  his  decision  making  ability.  The  provision  by  the  EC  of  the  information 
needed  to  reduce  uncertainty,  in  a  form  which  is  readily  assimilatabte  and  usable,  may  prove  the 
most  effective  strategy  in  making  the  pilot's  job  easier  and  safer.  The  provision,  in  some  form,  of 
explicit  Probabilistic  Information  at  the  Human-Computer  Interface  may  prove  one  such  method. 
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Summary 

Over  the  past  four  years  Cambridge  Consultants  have  been  conducting  a  series  of 
research  and  development  projects  on  behalf  of  the  Human  Engineering  Department  of 
the  Royal  Aircraft  Establishment.  These  projects  have  been  progressively  looking  at 
the  use  of  Artificial  Intelligence  techniques  within  aircraft,  first  from  the  point  of  view 
of  providing  appropriate  development  support  tools  and  now  of  tailoring  those  tools 
through  applications  work. 

In  this  paper  we  will  be  giving  an  overview  of  the  toolkit,  now  christened  MUSE,  and 
looking  at  some  of  the  applications  work  directly  using  MUSE,  or  related  to  this  area 
of  AI  and  combat  aircraft. 


A  Toolkit  for  Cockpit  Applications 

Back  in  early  1984  the  developers  of  A»  ../stems  who  wanted  to  engineer  real-time 
on-line  applications  were  faced  with  something  of  a  problem:  "serious"  AI  systems 
could  only  be  developed  on  large  expensive  AI  Workstations,  which  were  also  the 
run-time  environment.  These  workstations  had  never  been  designed  for  interfacing  to 
the  "real  world",  for  running  real-time  on-line  applications,  and  there  was  certainly  no 
way  they  could  be  flown.  This,  then,  was  the  environment  in  which  MUSE  was 
conceived,  and,  curiously,  it  is  one  that  hasn’t  changed  a  great  deal  to  this  day. 

The  design  goals  for  MUSE  were,  roughly: 

(1)  That  it  should  provide  a  development  environment  for  real-time,  continuously 
operating  AI  systems. 

(2)  That  it  should  interface  cleanly  with  other  systems,  both  hardware  and  software. 

(3)  That  it  should  provide  a  means  of  testing  such  systems  on-board  aircraft. 

The  architecture  of  the  toolkit  that  emerged  from  these  requirements  has  two  principle 
components,  a  development  environment  based  around  Sun  Workstations  and  a 
delivery  vehicle,  made  up  of  a  single-board  computer  to  which  the  application  software 
is  downloaded  or  programmed  into  ROMs  for  further  testing  in  an  embedded  form. 
The  development  environment,  as  is  usual  for  AI  programming,  provides  rapid 
prototyping  capabilities  through  an  editor,  incremental  compiler  and  debugging  tools. 
These  tools  extend  down,  as  far  as  possible,  to  the  target  machine  through  inclusion  of 
a  logging  facility  for  the  target  and  a  corresponding  log  browser  for  the  development 
system. 
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The  choice  of  language  facilities  for  the  toolkit  was  not  a  particularly  difficult  one.  A 
basic  language  was  needed  that  had  a  couple  of  essential  features  for  its  use  in  AI 
programming:  the  ability  to  treat  pieces  of  code  as  data,  and  great  flexibility  in  data 
typing,  along  with  a  compatibility  with  incremental  compilation,  essential  to  the  rapid 
prototyping  paradigm.  The  ultimate  choice  was  POP,  a  language  having  the  same 
capabilities  as  LISP  but  without  a  cumbersome  historic  syntax.  The  remainder  of  the 
language  facilities  are  somewhat  conventional:  extensions  to  the  basic  language  to 
support  object-oriented  programming  (of  the  Smalltalk  variety,  giving  the  new 
language  its  name  of  PopTalk),  frames,  demons,  access-oriented  procedure  calling, 
and,  of  course,  rule  based  programming.  Both  forward  chaining  (of  the  OPS-5  style) 
and  backward  chaining  forms  (as  per  Prolog)  can  be  mixed  as  appropriate  for  the 
application. 

The  target  machine  capability  is  obtained  using  a  Virtual  Machine  architecture,  i.e.  the 
components  of  the  language  package  each  have  a  run-time  interpreter  that  executes  the 
user's  programme  in  a  compiled  intermediate  code.  This  gives  compact  code  for 
memory  efficiency;  the  language  compilers  and  the  interpreter  are  written  in  C,  giving 
reasonable  portability  across  processors  and  good  run-time  performance. 

MUSE  has  other  features  that  make  it  appropriate  for  real-time  embedded  applications. 
The  most  significant  of  these  is  the  idea  of  modularity  within  the  application  software. 
Modularity  brings  multiple  benefits: 

•  By  partitioning  the  application  into  small  co-operating  modules  it  becomes  more 
easily  understood,  and  therefore  more  easily  designed,  verified  and  maintained. 
Since  the  scope  of  any  reasoning  module  is  limited,  side  effects  are  cut  down  and 
unexpected  behaviour  thereby  restrained. 

•  Each  module  can  be  implemented  with  the  most  appropriate  formalism  and 
without  incurring  die  run-time  overhead  of  unused  features.  This  applies  whether 
the  implementation  is  at  the  C  level,  PopTalk  level  or  Knowledge  Representation 
Language  level.  The  editing  and  compiling  facilities  of  the  MUSE  development 

nvironment  provide  a  coherent  means  of  developing  and  integrating  modules  at 
any  level  chosen. 

•  Perhaps  most  importantly,  modularity,  and  the  scheduling  facilities  that  go  with  it, 
provide  a  clean  mechanism  for  handling  interrupts.  An  agenda  is  provided  onto 
which  processes  (including  Knowledge  Sources,  rule  systems  and  procedures)  can 
be  placed  for  scheduling.  An  interrupt  may  result  in  a  high  priority  item  being 
placed  at  the  top  of  this  queue,  or  even  the  suspension  of  the  current  process  to 
deal  with  a  critical  event. 

•  De-composition  of  applications  to  co-operating  modules  is  the  first  step  towards 
concurrent  implementations,  Le.  large  grain  parallelism.  Since  the  Electronic  Crew 
member  carries  an  implicit  requirement  for  parallelism  (of  Data  Fusion,  Displays 
Management,  Weapons  Management,  Navigation,  Planning,  etc.)  this  is  an 
increasingly  important  feature  of  MUSE. 

MUSE  processes  have  been  designed  to  interface  to  the  outside  world  through  Data 
Channels.  MUSE  supports  tigh  speed  data  capture  by  providing  a  hierarchy  of  filtering 
operations  which  can  process  incoming  data  to  spot  important  events  which  should 
cause  asynchronous  scheduling  to  take  place.  Low  level  data  capture  is  performed  by  a 
simple  interrupt  driven  executive  (on  the  target  hardware)  which  is  intended  for  burst 
operation  up  to  lOOKbyte/s  region.  On  the  development  system,  MUSE  makes  use  of 
the  Unix  socket  mechanism,  allowing  clean  interfacing  to  other  processes,  for  example 
simulations,  even  across  networks.  Data  can  be  spliced  into  database  objects 
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filtering  procedures  are  available  to  allow  data  events  such  as  rapid  changes  of  values 
or  adverse  treads  to  be  detected  and  to  signal  an  interrupt  of  the  reasoning  system  via 
the  main  agenda  controller. 

MUSE  continues  to  evolve  in  the  light  of  applications  experience.  Two  major  areas  are 
currently  under  development:  a  "Multiple  Worlds"  facility  to  allow  an  application  to 
maintain  independent  interpretations  of  the  world  within  a  database,  and  a  framework 
for  temporal  reasoning,  of  particular  use  for  plan  representation.  As  well  as  these 
language  package  extensions,  alternatives  to  the  68000  processor  are  being  considered 
for  target  machines.  Of  particular  interest  here  is  die  transputer,  since  this  will  give  us 
a  ready  means  of  experimenting  with  true  concurrency. 

MUSE  has  given  us  the  enabling  technology  for  serious  practical  testing  of  AI 
applications  in  the  cockpit.  In  the  rest  of  this  paper  we  will  examine  where  this  is 
beginning  to  lead. 


On-line  Fault  Diagnosis 

Fault  diagnosis  is  one  of  the  most  popular  areas  for  applying  Expert  Systems  as  the 
rule-based  programming  paradigm  fits  the  diagnostic  method  so  well.  It  therefore 
seemed  a  natural  candidate  for  a  first  application  of  the  MUSE  toolkit  to  a  real-time 
problem.  The  specific  area  chosen  was  the  electrical  and  engine  subsystems  of  a 
helicopter,  the  aim  being  to  replace  the  Centralised  Warning  Panel  that  reports  fault 
symptoms  in  current  helicopters  with  a  warning  panel  that  reports  the  fault  status  of 
the  subsystems.  This  is  a  real  problem,  as  the  need  to  monitor  the  health  of  various 
systems  represents  a  continuous  load  on  the  pilot  It  is  also  a  task  that  is  often  cited  by 
pilots  themselves  as  being  something  they  would  like  to  delegate  to  an  automatic 
system. 

The  Central  Warning  Panel  (CWP)  consists  of  a  matrix  of  coloured  captions,  each 
corresponding  to  a  fault  symptom,  which  are  illuminated  on  the  presence  of  a 
symptom.  Some  fault  symptoms  are  not  flagged  by  the  CWP  and  are  only  indicated 
through  the  instruments  or  aircraft  response,  thus  pilots  must  regularly  scan  their 
aircraft’s  instrumentation  and  be  on  the  alert  for  irregular  responses  to  the  controls. 
The  pilot  diagnoses  the  fault  by  interpreting  the  symptoms  using  experience  built  up 
during  training  or  by  referring  to  a  set  of  ’flip-charts’.  On  diagnosing  a  fault,  the  pilot 
carries  out  the  appropriate  set  of  actions  to  recover  the  situation,  i.e.  recover  safe 
flying  parameters.  The  flip  charts  are  a  set  of  cards  listing  the  symptoms  and  recovery 
actions  for  each  fault 

Some  faults  can  initially  only  be  partially  diagnosed  and  require  the  failed  component 
to  be  put  under  test  before  a  full  diagnosis  can  be  reached.  For  instance,  a  pilot  may 
know  which  sub-system  has  failed,  but  has  to  carry  out  tests  on  it  to  narrow  down  the 
diagnosis  to  the  failed  component  of  the  sub-system.  Also,  in  the  case  of  intermittent 
faults,  once  the  component  has  been  placed  under  test  it  may  or  may  not  recover. 

Very  rarely  an  aircraft  will  have  multiple,  concurrent  faults  (these  are  usually  only 
experienced  by  a  pilot  under  simulator  conditions).When  they  do  occur,  the  pilot  must 
prioritise  them  according  to  their  seriousness  and  deal  with  the  most  fundamental  one 
first  These  fault  situations  are  extremely  stressful  to  the  pilot  and  are  made  worse  if 
he  is  carrying  out  a  complex  task,  e.g.  hovering  during  anti-tank  operations. 

The  Intelligent  Fault  Diagnosis  System  (IFDS)  operates,  currently,  from  simulated  data 
representing  analogue  and  digital  quantities,  e.g.  rotor  speeds,  engine  torques,  power 
rail  states,  switch  settings,  etc.  It  interacts  with  the  pilot  to  obtain  further  information. 
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for  example  requesting  him  to  select  or  deselect  a  piece  of  equipment,  in  order  to  take 
its  diagnoses  to  an  ultimate  conclusion.  At  all  times  it  supplies  the  pilot  with  as 
conclusive  a  diagnosis  as  it  can  given  the  information  available  to  it 

The  knowledge  on  which  to  base  the  fault  diagnosis  system  was  readily  accessible  in 
the  flip  chans  carried  by  the  pilot  and  from  the  pilots  themselves.  The  knowledge  is 
self-contained  and  static.  For  many  aircraft  applications  this  is  not  so.  In  ESM 
processing,  for  example,  emitter  characteristics  are  changing  all  the  time,  there  may 
well  even  be  emitters  appearing  in  time  of  war  that  have  never  been  observed  in 
peace-time. 

The  discipline  of  designing  the  IFDS  allowed  us  to  look  at  the  problems  of  building  an 
expen  system  that,  of  necessity,  had  a  very  meagre  interaction  with  the  pilot  It  was 
successful  because  the  information  needed  for  diagnosis  in  the  electrical  and  engine 
subsystems  is  available  from  sensors  rather  than  being  reliant  on  pilot  sensation  (of 
noise,  vibration  or  visual  effects).  The  IFDS  brought  us  face  to  face  with  one  of  the 
unfortunate  characteristics  of  intelligent  systems  -  interaction  with  them  often  requires 
a  dialogue  at  a  higher  level  than  button-pushes,  a  point  we  will  return  to  later. 


A1  in  Flight  Control  Systems 

CCL  were  engaged  by  RAE  Bedford  to  do  some  "Blue-Skies"  thinking  in  the  area  of 
AI  applied  to  Flight  Control  Systems  (FCS).  Aside  from  the  control  technology 
aspects,  we  began  to  consider  how  an  intelligent  FCS  would  interact  with  the  pilot.  It 
became  clear  that  there  are  several  ways  in  which  the  behaviour  of  the  FCS  could  be 
classified: 

Opportunistic 

There  are  occasions  when  an  FCS  can  behave  autonomously  in  selecting  modes. 
The  clearest  of  these  are  the  cases  where  switches  of  mode  will  be  practically 
indistinguishable  to  the  pilot,  but  will  result  in  fuel  savings  or  greater 
performance.  If  a  hierarchic  view  of  FCS  mode  structure  is  taken,  opportunistic 
mode  scheduling  can  be  used  at  the  lowest  levels  as  an  optimiser  within 
constraints  imposed  by  the  scheduling  of  higher  level  modalities. 

Reactive 

In  the  situation  where  a  pilot  takes  rapid  evasive  action,  for  example  reacting  to 
RWR  during  low-level  ingress,  the  FCS  will  need  to  react  to  the  sudden  change 
in  demands  from  high  stability  terrain  guidance  to  high  manoeuvrability  pilot 
control.  The  same  applies  when  performance  characteristics  of  the  aircraft  change, 
either  on  stores  release  or  for  failure  reconfiguration. 

Instructed 

This  is  the  most  obvious  of  cases,  and  the  most  conventional.  Changes  in  FCS 
mode  are  determined  by  the  pilot  and  explicitly  selected.  The  question  arises  as  to 
what  level  this  is  done  at,  i.e.  how  modes  are  organised  and  what  model  of  the 
FCS  the  pilot  is  presented  with,  since  an  FCS  may  have  several  hundred  inodes. 
Predictive  or  Pre-emptive 

In  this  case  confirmation  for  a  change  in  mode  is  given  by  the  pilot,  but  th' 
se lection  is  made  automatically  by  the  system  on  the  basis  of  a  prediction  of  w,._. 
the  pilot  will  wish  to  do  next.  Clearly,  if  other  types  of  scheduling  are  taking 
place  this  form  will  be  concerned  with  major  mode  changes,  corresponding  to 
changes  in  mission  phase  ffom  high  altitude  cruise,  to  low-level  ingress,  to  target 
acquisition,  to  attack  etc. 
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These  categories  of  behaviour  are  appropriate  to  other  types  of  onboard  systems  in 
which  we  might  tty  to  embed  intelligence.  Deployment  of  countermeasures,  for 
example,  would  fit  quite  comfortably  within  the  framework,  as  might  displays 
configuration. 


Pilot  Modelling 

One  of  our  current  projects  involving  MUSE  is  concerned  with  the  management  of  the 
interface  between  pilot  and  intelligent  systems,  specifically  with  the  development  of 
tools  for  an  aspect  of  pilot  modelling. 

It  is  clear  that  pilot  modelling  is  not  a  prerequisite  for  the  inclusion  of  intelligent 
systems  within  die  cockpit  Much  can  be  done  in  data  fusion,  systems  management, 
etc.,  without  needing  a  model  of  the  pilot’s  beliefs  or  likely  actions.  But  if  we  are  to 
build  systems  that  are  co-operative,  i.e.  acting  as  an  electronic  crew-member,  then  it  is 
equally  clear  that  such  systems  will  need  to  understand  the  pilot’s  actions  and  the 
motives  behind  those  actions. 

Our  initial  work  in  this  modelling  area  is  tackling  the  most  easily  accessible 
description  of  die  pilot’s  job,  the  mission  structure.  By  a  process  of  inference  from  the 
mission  plan  it  is  possible  to  build  up  a  reasonable  rough  representation  of  what  the 
pilot  will  be  doing  in  the  cockpit  throughout  a  mission.  This  description  provides  a 
context  for  interpreting  pilots’  actions  and  a  basis  for  predictive  scheduling  of  system 
activities. 
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l.  INTRODUCTION 

The  application  of  advanced  automation  end  Artificial  Intelligence  (AI)  systems  » the  cockpit  of  modem  commercial  and 
military  aircraft  holds  great  promise  for  the  extension  of  aircraft  capability  and  flight  safety.  However,  advances  in  cockpit 
automation  have  often  failed  to  meet  the  expectations  or  provide  the  advances  anticipated  by  the  technology.  One  reason  for 
this  shortfall  has  been  the  lack  of  integration  of  the  artificial  and  human  intelligences  in  the  development  of  aiding  for  the 
cockpit.  Over  the  last  several  years,  we  and  our  colleagues  have  explored  techniques  to  identify  requirements  for  intelligent 
pilot  aiding  systems,  and  have  defined  approaches  to  develop  them.  We  have  developed  an  architecture  as  a  basis  for  an 
Intelligent  Pilot  Assistant  (IP A).  Assuming  an  information  processing  structure  baaed  on  multiple  levels  of  information 
abstraction  and  model-based  reasoning,  we  provide  a  system  whereby  engineering,  cognitive  science  and  computer  science 
disciplines  can  be  applied  and  interact  through  a  consistent  interface.  We  have  developed  workstation-based  techniques  to 
explore  the  consequences  of  alternatives  in  automation  techniques  within  the  cockpit  and  to  explore  the  affect  of  varied 
assumptions  in  human  performance.  In  this  paper,  we  will  discuss  two  aspects  of  the  application  of  AJ  to  enhance 
aircrew  performance  and  expedite  advanced  cockpit  design: 


•  an  AI  aid  for  aircraft  system-failure  situation  assessment  and  response  selection,  and 

•  a  cockpit  design  aid  and  analytic  workstation  that  utilizes  AI  techniques 

1.1  Intelligent  Pilot's  Assistant  Function 

Advances  in  machine  intelligence  techniques  in  (liagnotia  have  yielded  expen  systems  in  which  the  machine  intelligence 
techniques  parallel  and  complement  human  information  processing  (1, 2).  The  IP  A  serves  as  an  interface  between  automatic 
and  human  reasoning  that  is  based  on  causal  models  of  aircraft  systems  represented  at  multiple  levels  of  abstraction.  The 
IP  A  formulates  responses  to  system  failure  based  on  diagnostic  expen-system  input  and  situation  assessment  techniques 
(3).  In  timeAjerfixmance  critical  portions  of  flight,  or  in  the  face  of  unique  multi-point  failures,  the  IPA  can  supply  fast- 
time  processing  and  bring  to  bear  multiple  sources  of  expertise  to  identify  the  cause  and  affect  of  failures  or  mission 
threatening  events.  We  are  guided  by  human  information  processing  models  to  determine  the  form  and  content  of  the 
displays  to  the  flight  crew.  We  will  describe  (he  human  information  processing  assumptions  that  are  the  basis  of  our 
interface  approach,  and  describe  the  application  of  this  processing  structure  to  an  intelligent  aid  for  aircrew  situation 
assessment 

12  Interactive  Design  Aid 

To  date,  the  major  emphasis  in  the  development  of  AI  systems  for  cockpit  aiding  has  been  on  the  availability  of  hardware 
and  software  systems  capable  of  contributing  to  (he  nominal  safety  and  executability  of  high-performance  aircraft  mission. 
There  has  been  little  effort  exploring  the  use  of  AI  technologies  to  aid  in  the  cockpit  design  process  itself  from  the 
perspective  of  operability  and  cognitive  demand  on  the  aircrew  who  must  use  these  systems.  As  a  consequence,  crew 
workloads  and  supervisory  tasks  have  steadily  increased  in  the  modem  cockpit .  We  have  explored  a  methodology  that 
includes  descriptions  of  aircraft  systems,  missions,  human  operating  characteristics,  and  formal  decomposition  of  procedures 
in  a  workstation  environment.  The  workstation  is  an  interactive  design-aid  that  allows  analysts/designer’s  to  explore  the 
impact  of  advanced  automation  from  a  specifically  human  performance,  goal-oriented  perspective. 

Z  HUMAN  INFORMATION  PROCESSING  STRUCTURE 

We  feel  that  selection  of  a  model  for  manAnachine  system  analysis  should  be  flexible  and  guided  by  the  purposes  of  the 
analyst  rather  than  a  theoretical  bias  or  technological  limitation  in  model  application  (4, 5).  To  that  end.  we  have  selected 
a  general  structure  proposed  by  Rassmusen  (6)  which  characterizes  human  information  processing  on  two  dimensions. 

First,  then  is  goal-directed  processing  moving  data  from  sensors  and  perceptual  systems  to  effectors  and  controls.  The 
paths  that  the  processing  can  take  is  described  by  a  second  dimension  based  on  the  level  of  abstraction  at  which 
information  processing  is  considered  to  occur.  Figure  1.  illustrates  the  dimensionality  of  the  model  and  the  paths  available 
for  information  processing.  The  figure  presents  states-of-knowledge  and  data  manipulation  that  move  the  operator  from  one 
state  to  another. 
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Figure  1.  Pilot  Information  Processing  Model 

Rasmussen  has  suggested  that  there  are  three  general  processing  strategies  that  can  be  distinguished  by  the  type  of 
information  with  which  the  human  interacts.  These  are  skid,  rule,  and  knowledge-based  response  generation. 

Skill-based  Response:  The  most  direct  and  fastest  response  generation  scheme  is  to  move  directly  from  sensors  to 
effectors,  a  kind  of  reflex  response  in  which  the  sensor  data  maps  immediately  to  an  operator  response  regardless  of 
circumstance  or  situation.  Skill  behavior  is  defined  as  a  compiled  sequence  of  familiar  subroutines.  The  links  between  the 
individual  or  primitive  units  of  skilled  behavior  can  be  influenced  and  selected  by  reference  to  the  environment  or  system 
state.  There  is  often  a  tradeoff  that  is  called  out  in  relation  to  skilled  behavior  in  which  the  granularity  of  the  primitive 
actions  is  traded  against  the  flexibility  of  the  skilled  response.  This  is  generally  considered  an  efficiency  tradeoff  in  which 
the  price  paid  lor  rapid  responae  a  loss  of  generality. 

Rule-Based  Responae:  Foliowing  the  arc  of  processing  in  Figure  1.,  one  moves  bom  perception  to  entity  and  state 
descriptions.  These  state  descriptions  are  representations  of  this  agent,  the  environment,  the  stale  of  plans  or  status  of 
cooperating  agents.  This  stage  along  with  an  assessment  of  the  significance  of  the  current  state  (situation  assessment) 
form  the  basis  for  rule-based  behavior.  The  information  extracted  bom  the  environment  and  system  status  sensors  has  been 
interpreted.  The  human  applies  pattern  matched  rules  to  the  interpreted  situation  in  order  to  provide  plans  for  action. 

Knowledge-based  Response:  Moving  toward  the  peak  of  the  processing  arc  (Figure  1. )  the  operator/pilot  is  considered  to 
be  engaged  in  knowledge-based  behavior.  This  is  the  (he  highest  level  of  abstraction  in  which  the  human  interacts  with  the 
environment.  In  knowledge  based  processing  the  operator  must  construct  an  internal  modei  of  the  environment  the 
mission,  the  aircraft  stale  and  the  current  goal-state.  The  pilot  can  then  monitor  the  condition  of  his  plan  and  respond  to 
anomaly  or  variation  in  that  plan  by  formulating  and  evaluating  options  for  action. 

Abstraction  and  induction  ate  the  processes  by  which  the  operator  moves  bom  the  skilled  interpretation  of  the  environment 
signals  to  the  knowledge  based  response  required  for  'deep  reasoning''  or  problem  solving.  Having  abstracted  a 
representation  of  the  carrem  aircraft  and  environment  state  the  operator  proceeds  to  determine  how  to  respond  to  that  state  in 
relation  to  his/her  goals  and  then  how  to  effect  that  plan.  This  implies  a  process  of  deduction.  The  induction  to  deduction 
path  can  be  circumvented  by  shortcut  paths  that  can  be  developed  by  training  or  provided  through  automated  aiding. 

This  is,  admittedly,  a  very  general  description  of  human  information  processing,  though  there  is  some  evidence  that  human 
diagnostic  behavior  in  emergency  situations  can  be  accurately  described  by  these  processing  distinctions  (7, 8).  We  are 
using  this  structure  in  two  ways.  First,  we  are  developing  an  aiding  system  for  situation/response-based  behavior  to  reduce 
the  time  writ  workload  required  to  identify  and  correctly  execute  response  procedure;  for  a  given  situation.  The  scope  of  this 
situation-response  model  is  rule-based  aiding.  That  is,  assistance  in  selection  and  execution  of  behavior  for  which  (he 
correspondence  between  situations  and  applicable  procedures  has  been  established  by  training  and  engineering-based 
analysis.  Situation-response  behavior  is  the  preferred  method  of  dealing  with  time -critical  flight  emergencies.  Accident 
analyses  suggest  that  in-flight  abstract  reasoning  may  shift  attention  from  flight-critical  tasks,  and  that  deep  reasoning  under 
sties*,  bom  necessarily  incomplete  information  and  incomplete  models,  can  produce  results  that  are  significantly,  and 
sometimes  fatally,  inferior  to  those  derivable  bom  engineering  studies,  experience,  and  simulation  experiment.  Second,  the 
framework  provides  a  point-of-contact  to  diagnostic  systems  that  function  at  multiple  levels  of  abstraction.  We  will 
discuss  those  applications  below. 
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3.  DIAGNOSTIC  SYSTEMS  AND  INTELLIGENT  PILOT  ASSISTANT 
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The  lolly  instantiated  architecture  for  an  Intelligent  Pilot  Assistant  includes  representation  of  the  aircraft,  the  aircrew,  the 
flight  environment,  the  flight  pLan/mission,  air  traffic  control,  and  procedurasAegulations/doctrine.  We  will  concentrate,  in 
this  paper,  on  interface  to  diagnostic  systems  work  which  we  are  performing  for  NASA-Langley  Research  Center.  Figure  2 
illustrates  the  interface  of  the  diagnostic  expert  systems  to  the  IP  A. 


3.1  Situation/Response 

We  have  designed  and  implemented  systems  to  aid  in  two  types  of  behavioral  responses  to  diagnosed  emergencies.  Aiding 
in  situationAesponse  behavior  has  been  approached  through  a  frame-based  representation.  There  is  a  situation  assessment 
process  in  which  situation  attributes  are  mapped  through  a  situation-type  taxonomy  to  a  current  situation  description. 
Response  procedures  are  selected  on  a  rule-based  reasoning  process  and  then  communicated  to  the  pilot.  This  procedure  is 
illustrated  in  response  to  a  Faultfinder  diagnosis  output  of  Figure  2.  The  process  is  detailed  and  expanded  in  Figure  3. 
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3.2  Causal  Model  Response 

Recent  developments  in  AI  diagnostic  systems  have  sought  to  improve  system  performance  by  adopting  techniques  of 
reasoning  at  varied  levels  of  abstraction  (9, 10).  We  have  been  supported  by  NASA-Langley  Research  Center  (Contract 
No.  NAS  I- 17333)  to  develop  an  aiding  system  that  can  take  advantage  of  diagnostic  reasoning  that  can  supply  based  on 
models  of  system  operation,  abstraction  of  principles  of  physical  causality  and  temporal  propagation  of  failures.  We  have 
structured  our  IPA  to  take  advantage  of  diagnostic  data  at  various  levels  of  abstractiou  and  to  aid  the  pilot  in  selection  of 
appropriate  response.  Abbott  mikes  the  point  that  graceful  degradation  of  diagnostic  activity  can  take  the  form  of  reduced 
specificity  rather  than  degraded  efficiency,  if  the  diagnostic  system  is  based  on  deep  functional  models  of  the  systems  being 
monitored  (9).  Using  a  process  of  status  abstraction  the  NASA  FaultFinder  system  produces  useful  symptomatic 
predictions  based  on  incomplete  or  ambiguous  data.  The  physical  reasoning  models  then  generate  hypotheses  of  failure 
propagation  and  simulate  those  failure  paths  checking  against  incoming  data  to  determine  which  is  the  appropriate 
hypothesis.  Our  IPA  also  reasons  about  situations  at  varied  levels  of  abstraction  and  takes  as  input  to  the  situation 
assessment  process  data  Grom  FaultFinder.  Figure  2  illustrates  the  general  process. 

We  reason  about  situations  and  suggest  response  based  on  a  three  tiered  level  of  representation ,  Boolean,  Qualitative,  and 
Quantitative  modelling.  We  can  reasoa  about  situation/respoose  requirements  base  on  sensor  level  data,  or  if  the  data  and 
processing  time  are  available,  based  an  qualitative  and  quantitative  models  of  aircraft  systems  and  flight  situations.  Figure 
3  illustrates  the  mapping  of  data  to  system  conditions  at  three  levels  of  abstraction  in  a  causal  model. 
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Figure  3.  Mapping  from  Physical  Systems  to  t  Multi-Level  Causal  Model 

A  cauaal  model  has  been  developed  to  support  this  more  sophisticated  diagnostic  process.  The  model  consists  of  four  parts 
in  a  frame-theoretic  representation.  These  are:  the  airplane  systems,  the  effectors,  the  farces  affected  and  aircraft  flight 
characteristics.  The  infereadng  mechanism  links  these  four  components  through  propagation  at  the  three  levels  of 
abstraction..  Forward  propagation  of  attribute  values  provide  a  simulation  of  fault  conditions  and  a  diagnosis  of  fault  cause. 
Backtracking  through  the  causal  model  provides  insight  to  alternative  paths  to  a  desired  system  state  and,  thereby,  suggests 
responses  to  achieve  those  states. 


Pilot  Vehicle  Interface:  Humans  or  automatic  controllers  require  some  information  on  the  state  of  whatever  it  is  they 
affecting  in  order  to  monitor  the  progress  of  the  response,  and  to  provide  the  feedback  necessary  to  implement  any  control 
laws  in  the  response.  Our  pilot  information  processing  model  suggests,  through  its  hierarchical  structure,  that  both  control 
and  feedback  be  available  at  multiple  levels  of  the  abstraction  hrenuchy.  The  particular  form  of  the  feedback  or  control 
should  be  geared  to  the  level  at  which  the  pilot  is  interacting  with  the  system.  In  order  to  support  operator  action,  two  sets 
of  information  need  to  be  supplied  to  the  operator.  First,  the  nature  of  the  sensor  to  rimarinn  translation  must  be  made 
dear.  Second,  the  nature  of  the  action  to  be  taken  must  be  pointed  out.  For  example,  if  the  IPA  has  taken  Faultfinder 
output  that  indicate  that  sensor  values  are  abnormal  in  a  way  that  unambiguously  requires  immediate  response  on  the  part 
of  the  pilot,  that  response  and  those  values  should  be  displayed  with  an  emergency  alert  status.  If  the  diagnostic  reasoning 
process  has  been  abstracted  to  reasoning  about  physical  propagation  of  a  fault  and  the  IPA  has  resolved  a  response,  display 
of  the  response  should  he  supplemented  with  a  display  of  the  physical  symptoms  identified  or  predicted  (perhaps  in  an 
iconic-schematic  format).  If  the  aiding  system  has  reached  an  impasse  in  response  selection,  a  trace  of  the  diagnostic 
reasoning  and  response  resolution  should  be  provided. 

4.  ANALYSTS  WORKSTATION  FOR  COCKPIT  AUTOMATION  DESIGN 

The  analysis  of  humanfsystem  performance  through  simulation  often  impose  constraints  on  the 
analyst/designer.  The  tradeoff  has  been  between  ease  of  operation  and  simulation  construction  on  the  one  hand 
and  the  degree  of  flexibility  and  generality  provided  by  the  tools  and  modelling  system  on  the  other.  It  is  with 
the  goal  of  mitigating  these  constraints  that  we  have  been  developing  a  set  of  modelling  tools  and  a 
methodology  for  their  application.  Simulation  of  humsn/rystem  operation  is  undertaken  to  provide  prediction 
of  system  performance  in  a  contest  that  is  controllable  by  the  designer.  We  believe  that  in  order  to  be  useful  a 
workstation-based  simulation  system  should  provide  the  following  features: 


•  A  coherent  and  integrated  framework  in  which  to  examine  the  interaction  of  particular  human  performance 
models  and  describe  the  interaction  between  the  operator  and  the  system  under  evaluation. 

•  Designer's  interface  tools  through  which  system  parameters,  model  parameters  and  task  requirements  can  be 
varied. 

•  Automatic  propagation  of  the  affects  of  changes  in  any  of  the  simulation  components,  activities,  or 
operational  events  throughout  the  system. 

•  Support  for  an  annotated  and  multi-perspective  representation  of  task  timelines. 

•  The  combination  of  a  bottom-up  constraint  implementation  of  the  system's  functions  with  a  top-down  goal 
decomposition  of  operator's  purposes. 

•  Multi-operator  •  multi-mission  capability. 

•  Insight  into  the  training  implication  of  a  given  human/machine  system  design.. 

•  Support  model  definition  at  a  level  of  detail  (hat  is  responsive  to  available  data  and  to  the  designer's 
requirements. workstation  environment  in  which  to  use  those  tools,  that 

Applications: 

We  have  been  working  to  provide  these  capabilities  by  providing  an  object-oriented  workstation/simulation 

environment  in  which  aircraft  designers  can  explore  the  impact  of  a  given  design  on  human/system  performance. 
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The  system  is  imptemetaed  in  Zetaliep  in  the  Symbolics  Lisp  Machine  environment  It  uses  the  Flavors 
object-oriented  system.  The  system  his  three  components:  A  simulation  driver,  libraries  of  object  descriptions, 

«nd  a  library  of  input  and  output  utilities.  The  simulation  driver  allows  a  user  to  run  sunulations  at  fixed  tune- 
increments  (driving  events  by  sending  time  signals  to  each  of  the  objects)  or  by  letting  the  occurrence  of  events 
drive  the  dock.  The  libraries  of  object  descriptions  include  descriptions  of  human  strategies,  procedures,  and 
tactics;  fine-grained  and  coarse-grained  descriptions  of  land-based  vehicles,  helicopters,  and  airplanes;  descriptions 
of  various  types  of  activities;  visual  auditory,  and  physical  models  of  human  capability  and  performance;  and  a 
variety  of  other  supporting  object  types.  The  input  and  display  capabilities  include  the  ability  to  display  the 
position  and  status  of  mottle  objects,  to  display  the  action  and  spawning  behavior  of  activities,  and  to  track  the 
changes  in  world  representations  of  cognitive  objects.  Aircraft,  their  subsystems,  cockpit  topology,  activities, 
missions,  and  human  operators  are  represented  as  objects  and  are  available  for  manipulation  by  the 
analystAtaigner  through  screen-based  tools  and  utilities. 

Of  particular  interest  is  simulatioa  and  prediction  of  performance  of  human  and  automated  systems  as  they 
engage  in  supervisory  and  cognitive  behavior.  The  basia  of  such  simulation  is  the  internal  and  updatable  world 
representation  of  active  cognitive  agents  in  the  simulatian  workstation.  The  interface  to  an  updatable  world 
representation  is  such  that  any  of  a  variety  of  data  sources  may  provide  it  with  data,  and  any  of  a  variety  of 
sources  may  request  data  bom  it.  Examples  of  sources  of  data  include  sensory  modules,  decision  modules,  and 
logical  modules.  At  present,  we  have  used  three  major  types  of  updatable  world  representation,  corresponding  to 
three  wayi  in  which  it  has  been  useful  to  simulate  the  storage  of  data.  One  type  stores  dam  keyed  to  the  object 
that  the  data  refin  to.  A  visual  scanner,  for  example,  that  determines  the  position  and  movement  vector  of  an 
aircraft  by  looking  at  a  radar  screen,  would  send  this  information  to  the  updatable  representation  in  the  form  of  a 
liar  containing  the  aircraft  object,  k  coordinates  and  vector,  the  source,  and  the  time.  A  second  type  of  updatable 
world  representation  stores  information  keyed  to  events.  When  information  is  given  to  the  world  representation, 
it  finds  the  event  or  events  for  which  the  information  is  relevant  and  associates  the  information  with  that  event 
or  events.  A  third  type  of  updatable  world  representation  stores  high-level  information  in  easily-accessible  local 
variables.  An  example  of  such  information  might  be  the  flight  plan  that  a  flight  crew  is  following.  Other 
types  of  updatable  world  representation  that  may  be  combined  with  these  include  forgetability,  limited 
information  storage  capacity,  and  stochastic  and  deterministic  information  degradation.  We  have  applied  this 
workstation  to  a  number  of  different  systems.  Including  advanced  fighter  design,  prototype  helicopter  cockpit 
design,  and  space  telsopentioo  control  station  design. 

3.  CONCLUSION 

We  have  suggested  that  A1  techniques  applied  through  architecture  for  an  Intelligent  Pilot's  Assistant  can  provide 
a  unified  and  integrated  approach  to  exploiting  computational  assistance  for  the  modem  aircrew.  At  the  same 
time  the  same  AI  architecture  in  a  simulation  and  workstation  environment  can  aid  analysis  and  designers  in 
assessing  the  impact  of  those  automation  alternatives. 
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Abstract 

Recently,  artificial  intelligence  and  advanced  automation 
technologies  have  matured  sufficiently  to  offer  considerable 
potential  for  assisting  the  pilot  -in  executing  complex 
cockpit  functions.  For  example,  sensor  fusion  algorithms  can 
be  used  to  provide  an  integrated  representation  of  the 
tactical  scenario,  and  expert  systems  can  act  as.  systems 
monitors,  advising  the  pilot  of  systems  status,  or 
recommending  courses  of  action.  At  the  same  time  advances  in 
control/display  technologies  such  as  full  color  flat  panels, 
helmet-mounted  displays  and  sights,  and  interactive  voice, 
provide  promising  candidates  for  the  design  of  an  advanced 
interface  between  the  pilot  and  the  avionics/weapons  suite. 

Currently  little  is  understood  about  how  to  optimally 
integrate  the  advances  being  achieved  in  computational 
processing,  knowledge  engineering,  and  automation  technology 
with  the  advances  being  achieved  in  control/display 
technologies.  The  critical  issue  of  allocating  integrated 
information  to  display  surfaces,  and  defining  appropriate 
operational  logics  for  system  control  must  be  resolved 
before  human  and  electronic  crew  members  can  effectively 
share  cockpit  responsibilities. 

1.0  Background 

The  anticipated  airborne  tactical  environment  of  the  post 
1990  time  frame  can  be  characterized  by  expanded  and  more 
intensive  operational  envelopes,  threats  of  increased 
numbers  and  severity,  large  volumes  of  Information,  and 
minimal  available  response  times. 

Effective  mission  execution  in  such  an  environment  depends 
upon  a  high  degree  of  pilot  awareness  of  the  tactical 
situation,  and  timely  and  efficient  responsiveness  to  the 
rapidly  changing  scenario.  This  requirement  imposes 
increased  demands  on  the  information  processing  and  decision 
making  capacities  of  the  pilot.  Bridging  the  gap  between 
demands  on  limited  human  resources  and  complex  and  dynamic 
operational  requirements  dictates  the  development  of  a 
pilot-vehicle  interface  that  1)  enhances  the  presentation 
and  utility  of  tactically  relevant  information,  and  2) 
facilitates  natural  and  efficient  pilot-system  interaction. 

Recent  advances  in  automation  technology  and  artificial 
intelligence  offer  substantial  promise  for  reducing  the 
pilot  workload  associated  with  extensive  information 
integration  and  interpretation.  Furthermore,  a  variety  of 
emerging  control  and  display  (C/D)  technologies  have 
demonstrated  capabilities  with  the  potential  for  enhancing 
the  interaction  between  system  and  pilot.  The  convergence  of 
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automation  technologies  with  emerging  C/D  technologies 
offers  the  advantage  of  interfacing  an  intelligent 
application  of  decision  aiding  with  a  natural,  flexible,  and 
efficient  pilot-vehicle  interface.  The  consequent  heightened 
information  transfer  between  pilot  and  system,  however  also 
increases  the  potential  for  information  and  task  overload. 
This  fallout  can  severely  attenuate  or  compromise 
prospective  technology  benefits.  The  crucial  challenge  for 
the  cockpit  designer  is  to  integrate  these  capabilities  to 
enhance  the  utilization  and  effectiveness  of  information. 
This  means  that  designers  must  address  issues  such  as: 
appropriate  allocation  of  operational  functions  to 
technologies,  minimizing  the  impact  of  technology 
limitations  on  mission  performance,  and  optimizing  the 
character,  quality,  flow,  and  priority  of  available 
information. 

2.0  The  Potential  of  Advanced  Automation  and  AI 

Current  trends  in  avionics  design  have  focused  on 
distributed  avionics  systems  such  as  the  Pave  Pillar 
architecture.  This  approach  carries  with  it  the  advantage  of 
extensively  processing  incoming  data  before  presenting  it  to 
the  pilot.  One  of  the  most  useful  exploitations  of  this 
advancement  is  the  integration  of  multi-source  "data”  to 
provide  coherent  and  relevant  "information"  to  the  pilot. 
(In  this  context,  data  refers  to  the  raw  output  from  onboard 
sensing  devices,  while  information  refers  to  some  useful 
Interpretation  of  that  data.)  These  applications  include  a 
host  of  sensor  fusion,  tactical  situation  assessment,  and 
decision  aiding  functions.  Cockpit  functions  which  have  been 
suggested  as  suitable  candidates  for  automation  include: 

pre-  and  real-time  mission  planning 
sensor  fusion 

threat  and  tactical  analysis 

kill  assessment 

sensor  management 

target  prioritization 

weapons  and  countermeasures  employment 

diagnostics  and  fault  detection 

fuel  management. 

Extensive  programs  sponsored  by  various  government  and 
industry  AI  laboratories  have  focused  on  the  development  of 
these  automation  applications  for  potential  infusion  in 
future  weapon  systems  (Hayes-Roth,  Hayes-Roth,  Shapiro,  and 
Westcourt,  1981;  Lowrance  and  Garvey  1983;  Baron  and 
Feehrer,  1985;  Garvey,  T.  1987).  The  results  of  the  efforts 
have  generally  provided  positive  evidence  for  the 
feasibility  of  employing  AI  techniques  for  executing  cockpit 
functions. 

3.0  The  Integration  of  Automation  in  Crew  Station 
Development 

While  the  development  of  automation  and  AI  algorithms  have 
continued  to  show,  promise  in  the  labs,  more  limited  success 
has  been  achieved  by  crewstation  designers,  in  understanding 
the  role  of  automation  and  AI  in  the  cockpit.  Outstanding 
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issues  such  as  the  Icind  and  level  of  information  to  present 
to  the  pilot,  and  the  appropriate  allocation  of 
responsibility  between  the  pilot  and  the  system  must  be 
addressed  before  automation  technology  can  be  effectively 
exploited  for  operational  functions.  Cockpit  designers 
currently  have  little  available  data  on  how  to  present 
results  of  expert  system  assessments,  when  and  where  a 
recommended  course  of  action  should  be  displayed  for  pilot 
concurrence  or  veto,  or  which  conditions  dictate  automatic 
execution  of  a  selected  course  of  action,  etc. 

The  potential  infusion  of  automation  and  AI  in  the  cockpit 
has  also  underscored  the  requirement  for  enhanced 
pilot-system  interaction.  Thus,  various  crewstation 
development  efforts  have  also  focused  on  emerging 
control/display  technologies  as  means  for  facilitating 
information  transfer  between  system  and  pilot.  Full  color 
flat  panels,  and  helmet  mounted  displays  (HMD's),  show 
promise  as  display  surfaces  for  providing  the  pilot  with 
processed  data  output,  while  interactive  voice  and  helmet 
mounted  sights  may  be  effective  for  controlling  information 
flow,  requesting  status  information,  and  designating 
priority  information.  Each  technology  however,  also  carries 
with  it  specific  limitations  which  impact  its  ultimate 
integration  in  an  operational  environment. 

The  current  challenge  for  crew  station  designers  therefore, 
is  the  appropriate  allocation  of  functions  between  pilot  and 
system,  and  the  effectively  distribution  of  the  consequent 
information  and  control  requirements  in  light  of 
control /display  technology  limits. 

In  some  instances  tne  automation  technologies  and  AI 
technologies  can  be  employed  to  create  more  meaningful  and 
interpretable  categories  of  information  ot  present  to  the 
pilot.  In  other  cases  automation  techniques  can  be 
effectively  employed  to  overcome  control/display 
limitations . 

The  re.,  ■'nder  of  this  paper  discusses  some  outstanding 
current  control/display  integration  issues  and  the  potential 
for  applying  advanced  automation  or  AI  to  mediate  current 
technology  limitations. 

3.1  Head-Down  Display  Capabilities  and  Limitations 

Full  color  flat  panel  displays  can  graphically  portray 
integrated  tactical  situation  information  derived  from 
processed  sensor  data.  The  objective  of  such  a  presentation 
is  to  provide  the  pilot  with  "situation  awareness",  that  is, 
knowledge  pertaining  to  the  geometric  relationship  between 
ownship,  potential  threats,  and  mutual  support.  While  the 
"processing"  required  to  represent  spatial  geometry  is 
feasible,  there  exists  a  number  of  "display”  limitations 
which  preclude  a  simple  depiction  of  this  geometry  on  a 
single  display  surface.  One  limitation  is  the  difficulty  of 
depicting  the  third  dimension,  or  vertical  separation  on  a 
two  dimensional  display  surface.  The  second  limitation  is 
display  size. 
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A  number  of  alternatives  have  been  proposed  for  presenting 
critical  vertical  separation  information.  They  include 
displaying  absolute  altitudes  in  digital  form  adjacent  to 
aircraft  symbols,  to  portrayal  of  specialized  vertical 
situation  displays,  to  depiction  via  a  perspective  grid. 
Which  mode  of  representation  (and  under  which  conditions)  is 
most  appropriate  for  workload  reduction  and  enhanced  mission 
performance  has  yet  to  be  empirically  determined.  Factors 
which  are  likely  to  influence  how  vertical  separation  cam  be 
optimally  portrayed  include  task  specific  requirements, 
cognitive  processing  demands,  degree  of  display  clutter, 
etc. 


3.2  Helmet  Mounted  Display  Issues  Capabilities  and 
Limitations 

A  helmet  mounted  HMD/HMS  has  the  potential  for  dramatically 
improving  the  operational  effectiveness  of  fighter  aircraft. 
While  there  is  little  precedent  for  use  of  this  technology 
in  fixed  wing  fighter  aircraft,  all  indications  point  to 
substantial  achievements  in  the  areas  of  optical  design, 
size  and  weight  reduction,  and  life  support  and  escape 
system  compatibility,  thus  making  the  integration  of  helmet 
technologies  a  reality  in  the  1990  time  frame. 

Three  primary  applications  have  been  noted  for  which  an 
HMD/HMS  integration  can  have  a  direct  and  significant 
impact:  1)  target  designation/weapons  employment,  2)  visual 
target  acquisition,  and  3)  attitude  awareness. 

Target  Designation .  Currently,  target  designation  is 
constrained  by  the  forward  field-of-view  (FOV)  of  the  HUD. 
An  HMD/HMS  increases  the  available  field-of-regard  (FOR)  for 
target  designation  to  the  entire  envelope  of  pilot’s  head 
movement  and  sensor  weapons  capability.  The  advantage  for 
operational  effectiveness  is  the  potential  for  off-boresight 
target  designation  without  compromising  other  aspects  of 
aircraft  employment . 

Target  Acquisition .  One  of  the  most  demanding 
perceptual /cognitive  functions  in  air-to-air  engagements  is 
achieving  line-of-sight  on  a  target  transitioning  from 
beyond  visual  range  (BVR)  to  within  visual  range  (WVR) .  An 
HMD/HMS  can  reduce  the  workload  associated  with  this  task  by 
indicating  via  a  reticle  ("reverse  cueing")  on  the  pilots 
visor,  the  position  corresponding  to  the  location  of 
"priority"  targets  in  space.  Furthermore,  when  targets  are 
not  within  the  pilot's  forward  FOV,  directional  vectors  can 
be  presented  to  indicate  the  azimuth  and  range  of 
approaching  targets. 

Attitude  Awareness.  Operations  at  night  and  in  adverse 
weather  can  result  in  loss  of  attitude  awareness  or  in 
unusual  aircraft  attitudes.  Attitude  reference  and  unusual 
attitude  recovery  symbology  projected  to  the  HMD  visor  can 
assist  the  pilot  in  maintaining  flight  control  in  low 
visibility  conditions. 
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Despite  the  enormous  potential  advantages  of  HMD's,  there 
are  a  number  of  limitations  to  projected  capabilities.  The 
most  significant  limitation  to  current  HMD  technology  is 
FOV.  Anticipated  FOV  for  the  1990  time  frame  is  between  20° 
to  30°.  The  consequence  of  this  limitation  is  the  potential 
for  significant  display  clutter  with  high  symbol  density. 
Critical  to  effective  use  of  the  HMD  is  "intelligent” 
selection  of  display  symbols  and  formats  based  on  current 
aircraft  state,  environmental  conditions,  pilot  intent,  etc. 
Algorithms  can  be  developed  which  automatically  select 
display  information  based  on  system  knowledge  of  these 
parameters.  Furthermore,  system  status  information  can  be 
used  to  enable  declutter  modes  without  the  requirement  for 
significant  pilot  intervention. 

Another  critical  integration  issue  impacting  the  potential 
operational  employment  of  HMD's  is  HMS  accuracy.  Tolerable 
windows  for  weapon  aiming  and  reticle  cueing  accuracies  must 
be  measured  and  specified.  Furthermore'  appropriate 
techniques  need  to  be  defined  for  managing  the 
interdependency  between  head  motion  and  aircraft  motion. 

Intelligent  use  of  pilot  and  system  status  information  can 
be  used  to  configure  current  HMD  format  appropriate  to 
immediate  situational  needs.  For  example,  head  position 
information  can  be  used  to  implement  a  "virtual  HDD”  thus 
replacing  HDD  symbology  during  off  axis  viewing. 

3.3  Interactive  Voice  Issues  Capabilities  and  Limitations 

Over  the  past  decade,  interactive  voice  technology  has  been 
progressively  viewed  as  having  the  potential  to  reduce  pilot 
workload.  Noted  advantages  of  interactive  voice  as  a  cockpit 
interface  include:  1)  offload  of  cognitive  tasks  from 
saturated  visual/spacial  resources  to  the  auditory 
processing  channel,  2)  the  facilitation  of  "eyes  out”, 
"hands  on”  operations,  and  3)  "natural"  and  "direct"  data 
access . 

This  potential  advantage  has  been  formalized  most  concisely 
by  Wickens  Multiple  Resource  theory  (Wickens,  Sandry,  and 
Vidulich,  1983) .  The  theory  proposes  that  the  workload 
associated  with  any  task  is  mediated  by  two  primary  factors, 
1)  the  compatibility  between  input  and  output  modalities, 
and  2)  the  degree  of  competition  among  limited  resources 
during  concurrent  or  time  shared  tasks.  Predictions  are  that 
1)  spatial  tasks  will  be  better  performed  when  mediated  by 
visual  input  and  manual  output,  and  that  verbal  tasks  will 
be  more  efficiently  performed  when  mediated  by  auditory 
input  and  speech  output,  and  that  2)  the  performance  of 
concurrent  tasks  will  be  easier  when  the  tasks  use  resources 
from  discrete  input  and  output  modalities. 

The  implication  for  cockpit  tasks  is  that  while  tasks  which 
are  spatial  in  nature,  (such  as  flying,  target  designation, 
etc.)  may  be  better  suited  to  visual  cues  and  manual 
responses,  other  functions  of  a  linguistic  nature  (eg.  data 
entry,  avionics  mode  selection,  etc.)  may  be  better 
accomplished  using  auditory/ speech  processing  resources.  A 
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further  implication  suggests  that  significant  workload 
reduction  could  be  accrued  by  careful  allocation  of 
concurrent  tasks  to  non-competitive  processing  modalities. 

Support  for  the  effectiveness  of  interactive  voice  in 
reducing  pilot  workload  has  been  provided  by  a  variety  of 
basic  and  applied  research  studies.  Results  of  a  number  of 
studies  comparing  voice  and  keyboard  data  entry  for  input 
time*  accuracy,  and  simultaneous  tracking  task  performance 
(Color,  Plummer,  Huff,  and  Hitchcock,  1977;  Skriver,  1979; 
Poock,  1980;  Poock,  1981;  Jay,  1981;  Ruess,  1982;  Simpson, 
Coler,  and  Huff,  1983;  Aretz,  1983;  Simpson,  et  al  1985; 
Beckett,  1986;  Szerszynski,  and  Van  Loo,  1987)  have  shown 
that  voice  input  provided  a  marked  advantage  with  respect  to 
secondary  task  performance.  That  is,  when  target  tracking  or 
flying  a  specified  profile  was  required  to  be  performed 
concurrently  with  data  entry,  there  was  a  significant 
positive  impact  of  voice  input. 

The  implication  of  this  for  cockpit  tasks  is  that  while 
certain  tasks  which  are  spatial  in  nature,  (such  as  flying, 
geometric  designation,  etc.)  are  better  suited  to  visual 
cues  and  manual  responses,  other  functions  of  a  linguistic 
nature  (eg.  data  entry,  avionics  mode  selection,  etc.)  may 
be  better  accomplished  using  auditory /speech  processing 
resources.  A  further  implication  suggests  that  significant 
workload  reduction  could  be  accrued  by  careful  allocation  of 
concurrent  tasks  to  non-competitive  processing  modalities. 

Achieving  these  advantages  in  an  operational  environment 
relies  on  a  robust  recognition  system,  while,  current  speech 
systems  vary  widely  in  recognition  accuracy  and  processing 
speed,  the  most  mature  have  been  demonstrated  to  perform  at 
about  98%-99%  accuracy  under  laboratory  conditions  (Simpson, 
et  al,  1985) .  Robust  performance  (98%)  of  speech  recognition 
systems  has  also  been  modestly  supported  by  field  studies 
(Poock  1980,  1981)  for  vocabulary  sizes  up  to  240  words. 
Noise,  which  at  one  time  posed  a  major  challenge  to  speech 
system  performance  has  been  successfully  overcome  through 
the  implementation  of  noise  cancellation  algorithms,  and 
noise  cancellation  microphones  (Coler,  1982;  Joost,  Moody, 
and  Rodman,  1986;  Szerszynski  and  Van  Loo,  1986) . 

Not  all  evidence  however,  points  to  optimism  with  respect  to 
the  cockpit  integration  of  interactive  voice.  First  of  all, 
there  is  a  lack  of  guidelines  for  associating  task 
characteristics  with  specific  processing  channels,  while 
some  tasks  are  clearly  dominated  by  a  discrete  channel, 
others  appear  to  utilize  multiple  channels  and  are  therefore 
difficult  to  assign  to  a  specific  input/output  mode.  In 
addition,  the  allocation  of  cockpit  functions  based  on  the 
assumption  of  resource  competition  is  extremely  dependent  on 
specific  cockpit  configuration,  mission  scenario,  and 
mission  phase.  Finally,  environmental  and  psychological 
factors  can  have  a  severe  impact  on  the  performance  of 
speech  recognition  systems.  Speaker  variability,  due  to 
stress,  acceleration,  fatigue,  etc.  can  severely  degrade 
both  system  performance  and  the  users  capability  to 
effectively  interact  with  it  (Hecker,  Stevens,  von  Bismark, 
and  Williams,  1968;  Porubcansky,  1984) . 
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Intelligent  processing  can  be  exploited  to  mediate  a  number 
of  the  integration  issues  associated  with  interactive  voice. 
For  example/  limitations  imposed  by  syntaxing  requirements 
can  be  attenuated  by  real  time  selection  of  syntax  based  on 
current  aircraft  state.  Furthermore,  flexibility  approaching 
natural  language  interaction  can  be  approached  by 
implementing  "wordspotting"  techniques  rather  than  strict 
vocabulary  syntaxes.  Another  advanced  processing  application 
for  mediating  cockpit  voice  integration  is  the  use  of  speech 
models  under  acceleration  and  under  stress  to  compensate  for 
changes  in  spectral  characteristics  which  may  degrade 
recognition  performance. 

4.0  Summmary 

The  infusion  of  advanced  processing  and  automation  has 
generated  a  need  for  a  more  effective  control/display 
interface.  While  a  number  of  emerging  technologies  have  been 
proposed  to  meet  this  need,  current  limitations  hamper  their 
direct  transition  to  the  cockpit.  At  the  same  time,  the  very 
capability  which  has  created  the  need,  can  be  exploited  to 
mediate  the  integration  of  new  controls  and  displays  thus 
attentuating  the  impact  of  technology  limitations.  The 
specific  implementation  approach  however,  must  yet  be 
defined.  Close  coordination  between  research  in  advanced 
automation  and  artificial  intelligence  and  research  in  crew 
system  designers  is  required  before  an  effective  integration 
is  achieved. 
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SUMMARY 

Much  of  tbs  HtxnnyEIectrcnic  Crew  liunmn  has  examined  tha  joint  Human/Computer  dedsx»  making.  This 
paper  takes  as  its  stanini  point  Rssnnisten  (1983)'s  ptopoamco.  that  the  control  systems  designer,  the  uperam  and  the 
control  computer  ate  oonaidaesd  as  cooperating  decision  malrsts  La.  mnch  of  the  decision  making  is  embedded  in  tha 
system  design.  The  proposition  is  adapted  hare  B  consider  the  HC,  the  EC,  and  the  project  design  ootatmmity  ae 
cooperahn*  deciaion  makers.  Thiipkperixaminea  the  implications  of  thij  proposition  for  the  design  proceea.  It 
introduces  the  concepts  of  'design  oomnnaiity’  and  'embedded  derisinna'.  with  a  airoplifieri  history  of  the  design 
process  evolution  to  date.  It  is  projected  that  Bsdhsse  guitar  sefatyorpsafttmenee  in  ftntae  systems,  name  deejaions 
wdt  hare  to  become  embedded.  with  more  knowledge  task  inao  the  cockpiL  Design  decuaon  making  is  than  examined 
m  assess  how  much  of  this  could  baeoan  embedded,  md  B  ideadfy  the  changes  leqnhed  in  the  design  process  to  allow 
this  to  happuL 


PREFACE 

About  ten  years  age  (Ref  1  \  it  was  pot  B  ms  that  ete  non  or  lass  loser  how  B  go  about  designing  e  traditional 
knobs  and  dials  aockpit- the  only  problem  nss  that  people  didn't  wmt  them  stqr  more,  they  wmaed  cockpits  with 
CRTs  in  them.  andwedkint  know  how  B  design  these.  Since  than,  than  hero  bean  major  advances  in  the 
availability  and  lice  of  immlieiri  rod  prototyping  tools,  formal  nntarwns.  front-end  analysis.  Human  Factors  research 
findings,  strategies  far  airerew  pamcipmipn  arc,  with  actmmmitaisintTeaaa  in  the  design  and  development  coat. 
However,  before  long  we  should  be  able  to  claim  that  we  have  the  CRT  cockpit  design  process  aider  comrol.  The  only 
problem  will  be  that. —  by  than,  people  won't  want  them  any  more,  they  will  Want  cockpits  with  huelligant  Support 
Systems  or  Electronic  Ciewmembeca  in  them.  At  the  moment  we  don't  know  how  b  go  about  designing  these.  This 
pep>r>v»mio«ew»w«rftb.>-.h»^».e..ii.d  — w  a.wt.m^u.1  m~-w  Much  of  the  change 

can  be  summarized  by  laying  that  in  the  move  m  CRT  cockpits,  we  have  had  to  implore  much  of  the  "how"  of  a 
mission;  for  the  H/EC  cockpit,  we  will  also  need  to  explore  the  "why". 


1.  THE  DESIGN  COMMUNITY 


fa  tha  Gmiting  case,  a  project  dteign  oomnsonity  could  be  the  global  population;  white  cockpit  review  meetings 
can  feel  like  that,  the  normal  busineaa  of  crewstabon  design  currently  takes  piaoe  in  a  large  but  tnwtegaabie  network  of 
personal  contacts,  with  a  certain  amowx  of  mandatory  md  contractual  structure  a  the  background.  The  limn  on 
manageability  we  rented  by  nuM-eka.  mnltl-natiional  working,  rethar  than  the  dwneada  of  tha  job  iasit 

The  teaaan  far  akantnung  the  design  community  concept  is  the  concern  (andregst)  that  if  the  procaea  of 
embedding  men  knowledge  md  decisions  is  B  contmua,  then  than  will  have  m  be  changaa  B  this  community.  If  the 
coat  of  this  embedding  procaea  is  to  be  contained,  dun  tha  Bow  of  information  and  knowledge  through  tbs  communiiy 
win  have  to  become  still  more  formalized,  md  more  tightly  managed. 

The  nature  of  the  design  community  rrotmd  a  project  depwida  upon  whethw  the  project  is  military  or  civil,  aimed 
it  performance  or  safety.  A  typical  list  of  the  community  far  a  military,  paifotmanca  oriented  project  would  include: 


-  cockpit  structure!  destines 
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-  aircraft  systems  designers 

-  Human  Fatten 

-  company  tenpikaa 

•  customer,  and  test  eatabliahmani  test  pilots 

-  test  establishment  (banal  feprtsantttivet 

-  mission  systems  specialists 

-  OR  spsdalists 

-  various  nsaochan  from  establishments,  perhaps  universities 

-  project  management  people  from  everywhere. 

-  it  soma  stage,  framing  and  training  simulator  people 

-  sometimes  MPT  (Manpower.  Personnel  and  Training)  specialists 

Future  systems  in  the  UK  am  likely  to  have  the  roles  of  all  these  parties  mapped  out  in  a  stakeholder  analysis  (Ref  2). 

The  number  of  people  involved  in  a  project  teaches  a  dnenahe  maximum  when  things  have  gone  wrong,  say  after 
an  accident.  Soma  interesting  Japeneee  trade  (Ref  3)  has  conducted  detailed  analytes  of  how  information  Sows 
through  such  a  wide  community.  «d  bow  it  varies  ovw  time  and  with  accident  type  (e.g.  Fig  1).  The  intention  here  ta 
to  use  it  earn  example  of  oveteR  information  flow,  because  the  esemples  grrwi  below  sse  vary  localized.  The 
contention  is  that  EC  will  affect  a  community  almost  ta  wide  ea  this. 

The  concept  of  "knowledge  as  t  group  prodmr ’  within  a  design  community  haa  been  brraetigiaad  Iqr  Poitou  (Ref 
4)  in  the  development  of  a  CAPCAM  syaaam,  Ha  ponita  out  that  knowiadga  is  both  ooOecnve  and  contingent,  and 
rums  up  as  follows  '..knowledge  at  work  in  m  industrial  outfit  it  naithar  homogeneous,  unified,  ccnsisumt, 
compiehenaive  nor  stable.  It  comas  from  dhrma  spatial  and  tampocei  origins,  it  it  mada  up  of  diipanee  puces 
distnbuled  among  ihe  mambaas  of  ihs  Srm.  according  to  tbeir  position  and  ids  in  the  drrinon  of  labour;  chunlts  of 
knowledge  am  waked  ate  with  impact  to  peculiar  if  not  dfvmgeet  conamsn,  maaeata  ad  goals;  btcaosa 

wvwricel  id  my  ie«»i.»i.l  tmmt  ^wwrtm|  an  djllhat  diydeaia.  the  hnriy  of  knowledge  it 

nevarcompiessnor  stable'  What  holds  tor  CADfCAM  hokla  (Or  (be  knowledge  that  goes  into  a  cockpit  in  human  and 
electronic  form. 


2.  EMBEDDED  DECISIONS  IN  DESIGN  .  SOME  EXAMPLES 


This  section  describes  tome  ntmplae  of  embedded  decieinm  in  pest  eal  present  synems.  They  win  dun  be 
used  to  eaptoru  cooperative  dednon  making  as  il  stsadi  and  a*  it  mg'  heve  to  become. 

2.1  ENGINE  LIMITS 


Tn  itae  r  ef  and  gnggleeflmnhe  eal  (Bela  rorkpit  rleriiime  ■  m  in  fine  Itmin  rmld  Trail  ha— i  been  left  to  die 
pilot;  no  markings  on  ihe  gauges,  inerieqnam  ineensrentnon.  esd  conuderabla  use  of  directly  sensed  information 
(smoka  and  tha  ameQ  at  burning  ad).  The  pilot  may  well  have  had  only  a  surface  modal'  of  the  engine  domain.  Indie 
HMD/CRT  cockpit,  ihe  more  highly  trained  pilot  svill  almost  certainly  have  a 'deep  modal  of  anginas  (and  the  one  he  i* 
(lying  with  in  paetacuUrX  end  the  cWpwiwgi  have  uud  their  knowledge  to  decide  how  best  to  help  tbepiloc  Their 
knowledge  u  uead  to  make  dariiinnt  tboot  the  donum  (U.  pradicriona  of  likely  engine  behavior*),  Kid  about  the  task 
(what  balp  the  pilot  is  lihaly  to  nead  wbm).  Thaaa  decision  appaar 

-  as  automation  (a*  auaomario  shoe  off  with  ovenpeead  on  (temp,  bast  sanags  for  Vow  noise  cBmb  outs), 

-  as  default  senates  (eg.  wiul  to  do  following  an  angina  fatbits  undar  a  pertacular  set  of  drcumatancaa).  esd 

-  m  indirarima  (s-g.  remaining  rangahene.  making  aaaumpoona  about  fuel  flow),  or 

-  at  lima  on  tsmpram  and  spaed,  (deciding  due  the  pilot  ahonld  not  exceed  these  values). 

For  EC  to  mmgs  daa  anginas  sad  D  keep  the  pilot  in  central,  tha  coUactive  decision  making  win  naad  to  change.  with 
yet  morn  deuiaaona  made  in  advance 

2.2  TORPEDO  RELEASE  (tacfasical  knowledge  obtainad  from  (Rsf  S» 


E*Jy  syasnns  had  moeh  of  tha  knowledge  embedded  in  manual!  and  training  cotnaa,  with  rebel  oonaol 
between  i  tsiucc  opstaior.  a  baliuopmr  cortrolkr  and  the  pilot  (far  MATCH),  or  jutt  in  die  head  of  the  pilot  (»ay  in  tha 
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Fiiny  Swordfish).  Knowledge  from  trills  orgwmanonj  sad  OR  rporiilim  continual  to  bs  pat  duo  HCs  by  ldnulaor 
tradidu  aid  in  ntort  complex  tacdce  mentals.  Sums  derision  about  optimum  tctpedo  perionnsnce  would  b» 
embedded  into  the  vihoui  settings  available  for  selection  (siy  far  rang!  or  depth).  Ccmpuahzad  nodes  nd  tnckift| 
computer*  h**«  onboddod  s  imlnnxie  of  deciaiom.  {or  example  shout  likely  tar|«  tnawm  capability.  ihout  how  to 
combine  das  tent  different  Mason,  shoot  how  boat  to  nltaae  s  weapon  in  relation  to  s  particular  target  isometry. 
Then  hive  been  numerous  attempts,  usually  unsuccessful,  to  build  decision  aids  to  add  to  the  knowledge  incorporated, 
and  o  aggregate  it  more  compiefaiy.  Because  of  computing  limits,  and  to  keep  the  HOcompuar  and  HC/Initructor 
dialogue  simple,  the  decisions  embedded  are  often  simplistic.  and  known  to  be  so  (e.g.  items  such  as  coolde-cuttar 
ranges,  weapon  acquisition  characterisnci).  There  is  a  strong  ccavicdon  that  IKBS  tschnology  win  creek  this  mu  and 
yield  large  perfccmaoca  benefits  with  an  H/EC  crew.  Achieving  this  will  have  implications  for  cooperative  decision 
maldng<HC,  EC,  Desip  Community). 


3.  COOPERATIVE  DECISION  MAKING  IN  DESIGN  AND  USAGE 


3.1  STATEMENT  OF  THE  PROBLEM 


The  bed  news  is  chat,  in  otr  operoon.  we  will  nsvar  find  tfts  philosopher's  stone.  We  will  never  find  a  process 
that  allows  us  to  deeip  softwero  in  l  perfectly  rational  way.  The  good  news  is  that  we  can  fake  it  We  cm  praam  our 
system  to  others  as  if  wo  had  bean  rational  designers,  aod  it  pays  to  pnrtnd  to  do  ao  doing  development  and 
maduainaica.'fRef  6).  Here.  Psmaa  and  Clamenta’  good  news  applies  so  the  documentation  of  CRT  cockpit 
technology.  This  section  examines  Ihe  potential  for  rationality  and  its  doctanemabon  for  H/EC  cockpits,  vising  mainly 
hum  the  collective  and  contingnx  nature  of  tha  knowiedga  involved.  TTawe  aspects  of  daatp  decision  making  are 

rryiwrUwxf 

.  dealing  with  rational  goals  is  difficult  in  both  dasign  aidinf  and  tactical  demasen  tiding. 

•  the  aanber  of  radonal  daaip  opdonp  generate  and  coneida  may  pcee  pea  difficulties. 

-  than  «e  eapeca  of  daatp  knowledge  that  may  not  yield  to  rental  approachea  or  documentation. 

3.1.1  RATIONAL  DESIGN  GOALS 

It  is  genvahy  aocepaed  that  the  H/EC  dialogue  wiU  have  to  be  tauin pinned  by  a  shared  undemanding  of  goals 
md  pnorinee,  and  indeed  Che  dialogue  is  likely  to  include  them  expikady.  Most  otxnputer -baaed  deeip  aids  include 
some  sort  of  pal  hinchy,  with  Weightings.  Attempts  to  evaluate  ratiofial  datip  aids,  or  to  put  them  mto  real  use  am 
thin  on  the  ground,  and  by  tnsllarp  mural  iftil  (Ref  7)  atthnup  thate  have  been  some  very  pronusing  syauxna  pot 
into  limited  use  (notably  the  wok  of  DDI  in  the  1977s).  Most  have  used  dedsioa  analytic  weightings,  similar  to  mvty 
cxparimvual  tactical  decjtiim  aide  The  laaaaa  tnempt  a  das  direction  is  Gob's  Desip  by  Objecrivee  (Ref  S.  and 
utuktlyiiig  much  of  tha  dunking  bt  Ref  2).  and  it  win  bt  iutertatiug  10  sea  how  daa  worka  on  Inge  systems.  Currant 
retevch  mao  knowledge  baaed  skit  to  dregs  decisions  finds  these  decisions  very  held  to  capture,  and  die  knowledge 
supporting  than  to  bspaiicuIaKiy  duties. 

3.1.2  RATIONAL  D€SK2f  OFITONS 

If  the  aim  of  hurotkieing  the  H/EC  is  aafacy,  than  ooa  avenue  is  to  pwaue  Sheridan  s  distinction  of  the  UNKarei 
the  UNK-UNK.  The  UNK  (unknown)  it  a  nasty  fnhne  conditioo  or  oombmadou  of  fathaea  dun  wo  daaip  for.  and 
strive  to  avoid.  The  UNK-UNK  is  a  circumstance  for  which  we  do  not  have  a  scenario,  and  the  main  reason  for 
having  the  HC.  Tha  first  approach  is  to  generate  aa  many  UNKs  as  possible,  and  thereby  reduce  the  UNK-UNICs. 
How  big  would  this  axptnaiun  have  to  be?  An  malysil  of  100  shipping  accklanta  (Rsf  9)  found  thu  tha  number  of  root 
causae  was  between  7  and  33,  frith  a  median  of  23.  The  median  nandnrof  gates  par  narwnrir  waa  12.  indicating  that 
das  number  of  reaps  between  tha  tanwir  (Troot)  causae  and  the  final  cccsstpiencs  was  fairty  large.  (Only  *  of  the  100 
wridanta  occurred  without  any  preceding  humoi  error).  An  expansion  of  dna  order  aounda  homndonaly  expensive, 
rad  difficuk  to  do.  Goo  reason  fbr  conducting  a  stakeholder  anaiyso,  and  having  a  daaip  community  with  divans 
intareaa  is  P  avoid  the 'groupthink1  (Ref  10)  inavitabla  in  acompnaad  datip  team.  Tha  pomta  to  neat  for  tha  future 
na  a)  ganararing  parados  and  anaantg  their  Ukshhood  it  a  ftigila  proems  (Raf  11).  b)  any  changes  to  tha  daaip 
procaae  mast  ancotaige  'vigflaptdaciaion  making',  radar  that  erprai  the  inflisanca  of  groupthink.  and  c)  the 
boiaidariaa  fbr  tkaignar-ian  tsapmihility  may  baconsa  a  legal  beeriegrored  in  paoduct  liabiliry  suits. 

3.1.3  ASPECTS  OF  DESIGN  KNOWLEDGE 
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Twitting  (Ref  12)  has  mad*  the  distinction  between  prudouial  tai  notmseive  jaeacriptioot;  a  prodattixl 
prescription  is  a  working  nik  of  thumb.  which  provide  guidance — lo  how  ID  idusve  t  certain  obkctive  (it  provides 
man  to  mis).  A  normative  ptsactipticn  «  a  judgemetmbom  whet  consent  las  gnod  or  lewfalcr  valid  conduct 
Writing  primaily  in  a  leg al  comm  he  is  inrnested  in  the  problems  0/  determining  the  scope  and  mealing  ol  ncrmaive 
preecriptione.  Th*  line  of  investigation  hare  it  the  extant  to  which  EC  can  make  use  of  nonnaive  pnacriptiont. 

A  major  investigation  into  distinctions  of  this  type  h«  been  made  by  the  philosopher  logon  Hebstmas.  Hii 
dunking  ippawx to  be  on  the  following  lina  (this  ie  t  goes  umptiflcetion  of  McCntiiy  (Ref  13)X  Actionembe 
considered  ten  1  number  of  viewpoints.  riptsswiting  different  momma  of  action.  or  different  sxpccq  of  a  anon. 
Purposive-rational  action  comprises  instrumental  action  and  itnugic  action.  Instrumental  iction  is  governed  by 
(  technicil  rala  boad  on  empxrh.il  knowledge.  In  every  ciM  thay  imply  empirical  predictions  about  oborvabk  emus. 

physical  or  aodsL*  Strategic  action  is  pert-technical,  pen-eocial  aid  ratal  to  the  derision  making  procadire,  end  is  at 
die  derision  ibecay  level,  tg.  (ha  choice  between  maxunin.  maxims*  esc,  aid  needs  supplementing  by  values  sad 
maxims.  Canunuakative  action  "is  governed  by  consensual  norms,  which  define  reciprocal  expectation  about 
behaviour  and  which  mast  be  uatastood  and  recognized  by  as  least  tarn  acting  subjects.  Social  norms  ate  enforced  by 
sanctions. . 

Violation  of  a  role  ha  a  diffdaut  consequence  according  to  die  type,  /nmwperent  behaviour  which  violates  valid 
teclmical  roles  or  strategies,  is  condemned  per  a  to  faihtra  through  lack  of  success;  dte  "pamshmsra"  is  built,  so  to 
speak,  inco  in  rebuff  by  reality.  Deviate  behaviour,  which  violasq  maennial  norms,  pcovoka  sanction  diet  ate 
connected  tridi  the  ruin  only  externally,  due  is  by  convention.  Leaned  rain  ef  papoeivoeationl  action  supply  us 
with  s*Ms.  insanialiaad  norms  takhpenreinHiysOiicniraa:  Skills  pot  us  into  a  position  to  aotve  problems,  motivation 
allow  n  to  follow  norms.' 


faqn—wnaal  action  meals  xyiiaa  eaqable  aedl  llfM  tariwnlwgy;  if  is  pwaaflile  si  n—  — .pfriral  Am  a.  xwnhaxt 
decision  about  ptedjetion  of  physical  behavioig.  and  than  can  be  embedded  as  surface  mlq  ora  deep  models. 

Strategic  action  is  rarely  astpticit  in  dodgi  a  preeanL  In  principla.it  is  pneeihla  to  caprueediis  with  KBS 
technology,  whether  it  would  be  practical  or  plaaaaBtfwnaaM  10  bateau.  Thsaaasapiadiatheeuihorhnexparienced 
ha  bawi  dw  nenewuy  to  make  explirir  the  variorr  threw  price  itias  ad  a>  have  Unas  svailahis  far  editing.  The  process 
of  capturing  ell  the  *whac-iraltwnativa  wig  he  time  nrwsnming.  but  nscaaaaqr  if  tha  H/EC  ie  10  haws  miopia.  Tha 
mayy  problem  ie  that  public  mnri ale  of  the  task  fall  a  long  way  rhertoi  the  actual  job,  and  a  yet  it  is  nor  obvious  how 
ID  make  good  the  gap  in  a  way  tint  acoornmodasa  da  UNK41NK,  or  otan  da  UNK. 

Ccnenunicetrve  action  sewn  to  be  die  newt  imarrable  if  sirrlmnkuw  we  pneehla  (md  dwy  are  supposed  to  be 
mandatory  for  expat  sysaeasX  dm  they  wffl  consist  of  chapan  and  verar  of  the  regulations.  or  the  monos  of  a 
meeting.*  in  which  it  was  speed  that—".  The  dUBcaUee  of  accommodating,  legitimizing  and  embedding  decision  of 
this  naan  have  beat  described  by  Hopkin  far  ATC  (Had  14)  *  A  product  of  working  far  won  is  the  development  of 
informal  gyaqi  practices  and  procedures,  tometiuwe  raterad  10  a  thottetnt,  which  oaenoOcts  here  evolved  to  deal 
with poticula Mode ofsimation  dm atiee  and  wan  twr  iwfgkolty  atvliegirt  h  for  lynaurterign  Such  practices 
evolve  for  good  leaBOns — some  existing  short-cuts  camaoc  ipparoady  be  openly  acknowledged  doing  AERA  2 
planning,  yet  it  teems  vital  dot  they  arm.  Daemons  atuet  be  taken  at  discard  or  to  sanction  diem.  If  cononHeta  would 
be  penalised  in  AERA  2  became  exatsig  dam -cuts  would  infringe  a  system  rule,  so  due  they  no  longer  adopt  them. 
A£RA  2  oouid  bring  twbtotim  radwr  than  inrreaa  m  traffic  handling  capacity.*  The  only  way  our  of  this  Hobson's 
choioa  is  arsdkalchenge  in  the  rtepone  rharwewittics  of  the  dasigl  oontnaiiMly  in  its  wiriest  itnee,  md  ea  explored  in 
Ret  2:  a  system  as  logl  a  tha  is  tafifceiy  to  natUfafC  raped  laaporaa  rates. 

Twining  hat  proposed  a  ditgnottio  modal  for  da  interpretation  of  normative  rules,  with  4  stages  examining; 

-  conditions  arinng  before  foe  tele  came  kneeniaenoe 
gfllnilrta  nel  arnei  aihentle  miking  trege 
leegrtnut  eeetntfog  star  iln  ceeetien  nf  itamli 
•  ipeciel  feaera  of  foe  parin' la  con. 

Whila  this  eppeae  10  be  eneecellaeneoda  hr  the  hananbaetproretion  of  (legeDralee.  the  proapect  of  conducting 
such  roiavtaigaim  hunch  ewey  dot  BCcatmihn  foe  nactaayhnaptairinnoa-UasioiradthotHndoui. 

Any  patimlw  derign  ■icriaon  may  txhfott  a  mixmts  of  all  dug  of  Habecmaa'  coegoria  (hence  the  use  of  the 
term  apoct).  However,  da  w**  of  embedding  e  decision,  or  the  pudMiiy  of  doing  this,  depends  on  the  mix.  The 
usajniataaieBidhioasIly  foil  of  poBticst  decisions  mqqtismkng  anchor  el  ones.  This  very  dclicscc  arse  will  have 
to  be  lead  bos  if  H/GC  is  to  work  a  ary  oomprahamva  manor.  The  examples  dm  follow  as  at  attempt  to  look  at  the 
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coop— he  decision  nuking  now  aid  in  lha  future. 


3.2  EXAMPLES 
3.2.1  ENGINE  LIMITS 

The  author1!  attempts  to  obtain  m  imdars tending  of  angina  Units  in  term  of  instrumental  action  mat  with 
atomwalling.  It  bacana  daar  that  dun  warn  other  forces  u  work.  These  an  shown  in  Fig  2.  The  question  is  -  how  is 
ECtotaaowwhan(t)hscsnaxeasdlsniis?  Furfur,  will  (tjhebn  any  <tas  if  (a)ha  can't? 

3.22  TORPEDO  RELEASE 

The  tranaforraatiana,  biodcagae  and  probiame  of  incorpotsang  a  full  union  marling  of  torpedo  release 
requritneatts  mao  cunerat  systems  no  shown  in  Pip  3  and  4.  The  diffirqltua  of  overcoming  these  aaam  formidable.  Do 
era  have  so? 

CONCLUSIONS 

Decision  making  is  shwed  between  HC,  EC  and  the  design  community. 

To  achieve  the parfbammee  or  safety  benefits  hoped  far,  the  dasipirtarialnn  making  enB  have  to  change. 

Mucfacn  rf  Tfs~  li~isi—  Iiiiilini  in  ftuign  is  tint  rlely  tirhnirtj-  ft  V*  1 - ruse  ~liin  — ill  Vi  ~ntj  klTIuth 
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TEP8T  Mg)  AWABKHE8S  IN  HUMAH-KLECTRCBIC  CREW  TKAIgjORK 


R.M.  TAYLOR 

Royal  Air  Fore*  Institute  of  Aviation  Medicine 
Famborough,  Hampshire,  UK 

"Mo  lesson  sasas  to  ba  so  deeply  inculcated  by  the  experience  of  life  aa 
that  you  should  never  trust  experta".  Salisbury,  Latter  to  Lytton,  1877. 

i .  nrrsooocTicw 

The  purpose  of  this  paper  is  to  examine  the  relevance  of  trust  and  awareness 
for  teamwork  in  the  Human-Electronic  Crew.  The  concept  of  the  Electronic 
Crewmember  or  EC  has  been  recently  extended  to  include  more  clearly  the  ability 
"to  make  decisions  that  may  be  critical  to  mission  success  and  survivability" 
(Ref  1).  This  indicates  fresh  optimism  about  the  potential  applicability  of 
Artificial  Intelligence  (AI) /Knowledge-Based  Systems  (KBS)  technology  for 
assisting  decisions  in  uncertainty,  when  the  outcome  is  not  definitely  known  or 
knowable.  At  present,  judgements  in  uncertainty  are  an  entirely  human  function. 
In  Courts  of  Law,  Juries  are  required  to  make  judgements  of  guilt  only  if 
certain  beyond  any  reasonable  doubt.  War,  of  course,  is  "the  province  of 
uncertainty".  A  functionally  effective  relationship  between  the  human  (pilot) 
and  electronic  crewmembers  has  been  characterised  aa  a  synergistic  partnership 
based  on  teamwork,  grouping  together  to  do  the  jobs  that  one  person  can't  do 
alone.  Key  decisions  in  uncertainty  could  involve  both  crew-members,  with  EC 
providing  advice  and  decision-support  and,  if  necessary,  making  decisions 
autonomous ly  with  both  active  and  implied  pilot  consent.  The  important  question 
1st  Bow  can  we  make  this  team  work? 

Trust  and  awareness  have  been  postulated  to  be  essential  ingredients  for 
effective  teamwork  in  the  Human-Electronic  Crew.  Improved  "situational 
awareness"  is  a  major  design  objective  for  future  Intelligent  Systems  (e.g- 
USAF's  Pilot's  Associate  Programme).  It  can  be  argued  that  awareness  is 
necessary  if  the  pilot  is  to  make  conscious  choices  and  act  adaptively  when 
dealing  with  uncertainty;  awareness  of  performance  is  not  necessarily  Involved 
In  skilled,  automatic  behaviour  where  there  is  no  uncertainty  and  no  choice  (Ref 
2).  Similar  "awareness"  will  be  necessary  for  the  EC  to  act  flexibly  and 
adaptively;  to  learn,  change  and  evolve  in  an  humanistic  "intelligent"  manner. 
Some  common  awareness  and  knowledge  is  essential  for  effective  teamwork.  But 
there  Is  uncertainty  about  the  extent  to  which  all  levels  of  knowledge  and 
awareness  need  to  be  commonly  held  or  shared  between  team  members.  When 
functions  and  tasks  are  distributed,  and  knowledge  and  awareness  are  divided, 
trust  between  each  partner  becomes  an  essential  feature  of  successful  teamwork. 
Trust  will  be  necessary  if  the  pilot  is  to  rely  on  the  EC  for  assistance  in 
decisions  critical  to  mission  success  and  survivability,  particularly  with 
automatic  task  allocation  by  implied  consent.  If  distrust  exists,  rightly  or 
wrongly,  the  full  potential  of  the  partnership  will  not  be  realised. 

Both  trust  and  awareness  are  abstract  concepts.  They  may  have  behavioural 
consequences;  but  they  are  not  tangible  experiences  that  can  be  observed  and 
measured  directly,  implementation  of  the  requirements  for  trust  and  awareness 
in  the  design  of  future  Human-Electronic  Crew  Systems  could  be  facilitated  by  a 
clearer  understanding  of  the  factors  affecting  trust  and  awareness  in  current 
aircrew  operations.  What  follows  briefly  describes  recent  IAM  studies  using  the 
Personal  Construct  System/  Repertory  Grid  Technique  to  investigate  how  aircrew 
understand  or  construe  "Awareness"  and  "Trust". 

2.AMAUU88  STUDY 

The  study  of  Situational  Awareness  (SA)  involved  Interviews  with  34  RAF  test 
aircrew,  conducted  in  three  phases;  1)  Scenario  Generation;  2)  Construct 
Elicitation;  3)  Construct  Validation.  At  first,  descriptions  of  flight 
scenarios  involving  SA  were  obtained  from  10  test  aircrew  at  RAF  Famborough  and 
Bedford  based  on  the  following  agreed  working  definition  of  SA:  "Situational 
awareness  Is  the  knowledge,  cognition  and  anticipation  of  events,  factors  and 
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variable*  affecting  the  safe,  expedient  and  affective  conduct  of  the  mission" . 
The  43  SA  scenario*  obtained  were  reduced  to  a  set  of  29  familiar  generic 
example* ,  of  which  the  following  are  typical: 

■bather  Approach:  Low  Awareness-  Approaching  to  land  at  an  unfamiliar 
airfield,  in  poor  weather,  in  an  unfamiliar  aircraft  fitted  with  poor  handling 
qualities  and  display*. 

Cembet/Good  Visibility:  High  Awareness,  in  air  combat,  you  are  behind  your 
opponent  and  over  a  familiar  area  with  good  horizon  and  height  cues. 

Next,  the  29  selected  scenarios  were  presented  to  14  test  aircrew  at  RAF 
Boscoabe  Down  to  elicit  SA  constructs.  Each  construct  was  elicited  using  the 
triadic  method  of  scenario  presentation.  All  29  scenarios  were  rated  on  a  7- 
point  scale  of  the  elicited  construct  dimension.  A  total  of  44  SA  construct 
dimensions  with  associated  scenario  ratings  were  obtained  in  this  way.  Principal 
components  analysis  indicated  that  4  factors  accounted  for  65%  of  the  total 
variability  in  the  data.  The  2  major  components,  contributing  30%  and  21%  of 
the  variability  were  dominated  by  Situational,  Informational  and  Attentional 
constructs.  Guided  by  this  analysis,  10  generic  constructs  were  selected  for 
further  evaluation. 

In  the  validation  phase,  the  10  constructs  and  29  scenarios  were  presented  to 
10  test  aircrew  at  RAF  Farnborough  for  scenario/ construct  rating.  The  29 
scenarios  were  split  into  two  arbitrary  groups.  Five  aircrew  rated  each  group, 
giving  two  independent  sets  of  data.  Statistical  analysis  showed  similar  data 
structures.  Both  data  sets  contained  a  component  loading  on  constructs 
concerning  Understand! ng  of  the  Situation  (Information  Quantity,  Quality  and 
Familiarity) .  Two  further  groups  of  constructs  distinguished  between 
situational  factors  placing  Demands  am  Attentions!  Resources  (Instability, 
Complexity,  Variability)  and  aspects  of  the  Supply  of  Attentional  Resources 
(Arousal,  Concentration  and  Division  of  Attention,  Spare  Capacity).  Ratings  for 
the  Heather  Approach  and  Combat/Good  Visibility  scenarios  are  illustrated  in 
Figs  1  and  2. 
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Quantification  in  each  of  these  three  domains  is  needed  for  a  comprehensive 
measurement  of  aircrew  Situational  Awareness.  Measurement  of  SA  say  provide  a 
useful  adjunct  or  alternative  to  workload  estimation  when  improving  awareness  is 
an  important  design  objective.  For  real-time  applications,  as  opposed  to 
imaginary  prospective  studies,  a  relatively  un -obtrusive  approach  would  be  to 
rate  Attentional  Demand,  Supply  and  Understanding  as  Low,  Medium  or  High,  as  in 
SHAT  workload  measurement,  with  analysis  by  conjoint  scaling  procedures.  An 
appropriate  acronym  would  be  TART  for  Taylor  Awareness  Rating  Technique. 
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Th»  implications  tor  teamwork  in  tha  Human-Electronic  Craw  arc  that  th«  EC  can 
enhance  pilot  Situational  Awareness  in  three  ways: 

1)  Control  Mm*  on  AStanV  tonal  Raeoaross.  Thia  can  bo  achieved  by  EC 
accepting  unwanted  workload,  fusing  data,  reducing  uneortainty, 

2)  MCMM  tha  (apply  of  tttMtloool  IMoonrooo.  sc  can  achieve  thia  in  several 
ways ;  Prioritising  and  cooing  tasks  to  obtain  optimum  at taut ion- allocation 
Strategy  in  aeeordanca  with  aiaaion  goala  and  objectives;  Organising  tha 
structure  of  caaka  to  exploit  tha  aval  labia  resource  modalities ;  Maintaining 
pilot  inrolvaront  and  activity  at  tha  option*  level  tor  resource  availability. 

3)  Tnpania  Understanding.  Mathoda  by  which  SC  can  iaprova  pilot  undaratanding 
includa:  Praaanting  Information  in  cognitivaly  compatible  forma  (3-D  sound  and 
pictorial  displays);  Making  aecaaaibla  and  abating  a  widar-knowladga  baa a 
through  knowledge  cowan ication/dialogua  techniques  auch  aa  lntarrogntion, 
explanation  and  critiquing;  Extending  tha  pilot'a  relevant  axparianca  by 
alanlation  training  through  gaming  and  mioaion  pra-viaw  facilitiaa. 


the  Trust  atudy  involved  interviowa  with  50  operational  aircrew  on  the  two- 
aeat  Tornado  SKI  aircraft,  following  a  contracted  veraion  of  the  8A  atudy 
procedure.  Brief  daecriptiona  of  12  tactical  daciaion-making  acanarioa 
involving  Trust  ware  obtained  from  S  aircrew  from  Do  27  Tornado  Squadron,  RAF 
Merhem.  Six  scenarios  concerned  navigator  decisions  and  6  concerned  Pilot 
decisions,  ell  Made  without  consultation  with  the  second  crew-member,  in  each 
Filot/Nav  decision  category,  3  scenarios  were  described  aa  "High  Trust"  and  3  as 
"Low  Crttst”.  Bach  description  wee  constructed  to  infar  or  contain  specific 
references  to  the  information  evaluated,  to  tha  alternatives  considered  and  to 
the  choice  of  action  or  Inaction  selected.  The  following  ere  typical  examples : 
cram  SttWUBi  tlUR  MCTIWM/MS  uun.  Plying  low-leval,  with  an  mammy 
approaching  unseen  on  starboard  beam,  on  hearing  e  "counter  starboard”  call 
from  e  baddy  aircraft,  without  consultation,  tha  pilot  decides  to  break  port. 
OCBMAMO  EJECTXOMi  MAPTfUTOS  OmZXSKM/WUM  MR.  With  tba  aircraft  in  a  dive, 
end  the  Pilot  not  responding  to  "recover"  inputs,  possibly  suffering  target 
fixation,  and  with  tha  ejection  seat  switch  sat  to  'both',  the  Hav  evaluates 
the  possibility  of  ground  impact,  lack  of  time,  ground  proximity  and  aircraft 
attitude,  and  chooses  to  eject  rather  than  to  taka  no  action. 

The  12  selected  decision  scenarios  were  re-presented  to  the  8  RAF  Martian 
aircrew,  using  the  triadic  method,  to  elicit  constructs  that  were  important  for 
Trust.  Twelve  constructs  emerged  in  this  way.  Bight  potentially  relevant 
constructs  were  added,  including  Dana ad  for  Trust  and  Actual  Trust  ( Supply ) . 

Next,  tha  12  decision  scenarios  and  20  Trust  constructs  wars  presented  to  42 
Tornado  eiraraw  et  RAP  Leerbrueh  and  RAF  Brugge n.  Eighteen  Navigator*  rated  tha 
Pilot  decisions  on  Trust  constructs  and  tha  Navigator  decisions  on  Awareness 
constructs.  Twenty-four  Pilots  gave  Trust  and  Awareness  ratings  on  the 
Navigator  end  Pilot  decisions.  Principal  co-ordinates  analysis  indicated  that 
the  Awareness  ratings  had  a  similar  structure  to  that  obtained  in  the  SA 
construct  study,  with  three  components  accounting  tor  60%  of  the  variance, 
corresponding  to  Attantionel  Demand,  Supply  and  Understanding.  Analysis  of  tha 
Trust  date  showed  that  5  components  accounted  for  approximately  65%  of  the 
variability  in  the  ratings.  The  3  major  Pilot  Trust  components  obtained  high 
loadings  aa  constructs  related  to  Risk,  Judgement  and  Doubt.  The  2  major 
Navigator  Trust  components  had  high  loadings  on  Judgaownt  and  Doubt  related 
constructs.  Risk  related  constructs  loaded  highly  on  the  2  minor  Rev 
components.  The  component  constructs  end  Trust  leadings  are  summarised  below  in 
Table  1  with  the  constructs  listed  in  approximate  order  of  component  loading. 

It  can  ba  seen  from  the  Trust  loadings  that  whereas  Demand  tor  Trust  is 
related  to  the  perception  of  Risk,  Supply  of  Trust  is  related  to  the  level  of 
Judgsmsn t/AWar snSas  end  Uncertainty /Doubt.  Demand  for  TTuet  generally  exceeded 
Supply .  The  shortage  in  supplied  Trust  wee  greatest  tor  the  Pilot  decision  not 
to  carry  out  e  low  level  weather  short  when  the  Navigator  considered  the 
conditions  unsafe  to  continue  (Demand  X  m  6.05;  Supply  X  -  3.05)  end  in  the 
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Countar  Starboard  scenario  daacribad  sar liar  whan  tha  Pilot's  dacision  to  break 
port  want  against  aspects tions  (Denand  I  »  5.79»  Supply  *  »  4.17).  Actual  Trust 
was  highest  Cor  tha  Pilot  dacision  to  break  right/laft  in  response  to  an  SW 
ai sails  warning  ( Basis nd  t  •  5.84;  Supply  X  -  5.79)  and  for  the  Haw's  dacision  to 
rnmmanl  eject,  described  earlier  (Deaand  *  ■  6.50;  Supply  X  -  5.38).  The 
individual  Trust  and  Awareness  ratings  for  the  Counter  Starboard  and  Coamand 
Ejection  scenarios  are  illustrated  in  Pigs  3  and  4. 


The  implications  for  teamwork  in  the  Buaan-Electronic  crew  are  that  SC  can 
enhance  Trust  in  the  following  ways: 

1)  Control  Demand  fur  Treat.  This  can  be  achieved  by  Minimising  the  risk  in 
asking  decisions  and  the  negative  impact  on  survivability  i  by  earl  el.  sing 
recoverability  after  unsuccessful  decisions »  and  by  assuring  consistency  with 
mission  objectives.  The  embodiment  within  SC  of  pilot  intentions,  agreed 
goals,  mission  objectives,  governing  rules  and  rules  of  engagement, 
exemplified  by  Asimov's  Three  Laws  of  Robotics  (Ref  1),  would  provide  the 
logical  structure  for  the  behaviour  of  each  partner  in  a  rational,  consistent 
and  reliable  rather  than  arbitrary  manner.  Governing  rules  are  the  key  to 
minimising  risk  and  reducing  the  demand  for  Trust,  particularly  if  SC  is  to  be 
allowed  to  make  decisions  autonomously.  Time  pressure  could  be  reduced  by  SC 
anticipating  decision  and  action  requirements. 

2)  Tdp mss  Supply  of  Treat.  SC  can  achieve  this  in  two  ways.  Firstly,  by 
reducing  the  uncertainty  and  doubt  in  decision-making,  thereby  increasing  tha 
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confidence  and  probability  of  a  aucoaaaful  outcoms  ■  Sacondly ,  by  enhancing 
tha  quality  of  judgement,  assessment,  awaranaaa  and  knowledge  involved  in 
daoiaiona.  BC  can  raduca  uneartainty  by  limiting  tha  ntwbar  of  alternatives 
undar  conaidaration  and  by  providing  aatiaataa  of  utilities  ,  rlaka  and  outcome 
probabilitiaa  (Raf  3) .  Practical  aathoda  by  which  1C  can  anhanca  Pilot 
awaranaaa  wara  idantifiad  earlier.  applying  judgement  raquiraa  knowledge 
about  what  to  do  with  information  (meta-knowledge) .  Judgaaant  and  tha  aupply 
of  Truat  would  ba  furthar  anhancad  if,  for  inatanca,  BC  wara  abla  to  aaaiat  in 
problaa  racognition  and  formulation,  in  tha  ganaration  and  avaluation  of 
hypotbaaaa  and  decision-making  atratagy,  and  in  tha  avaluation  of  daoiaiona 
uaing  feedback.  This  nay  raquiro  AX/KB8  technology  capable  of  handling  more 
complex  hauriatical  propoaitiona  than  "If,  than"  a tat amenta. 


4.RULI 

Tha  aupply  of  truat  and  attentional  raaourcaa  should  not  exceed  tha  demand  nor 
tha  demand  exceed  tha  supply.  Tha  former  loads  to  gullibility  and  boredom,  tha 
latter  to  suspicion  and  error.  According  to  our  sura  pragmatic  aircrew,  Truat 
is  "being  abla  to  get  on  with  your  own  job  without  worrying  about  what  tha  other 
crow  member  ia  doing"  ...  "If  it  doesn't  work,  it  can  kill  you;  if  it  does  work, 
it  can  save  your  life"  ...  "Blind  Truat  is  dumb".  All  agreed  that  "real"  Trust 
(i.o.  supplied)  is  built  up  through  communication  and  experience  as  a  successful 
team.  Proving  that  BC  deaerveo  to  be  treated  ia  the  challenge.  Truat  is  proven 
by  resolving  doubt  through  knowledge,  communication  and  awaranaaa.  Blind  truat 
is  a  naive  strategy,  implying  an  assumption  of  certainty  without  knowladga  and 
awaranaaa.  The  only  certainty  is  that  nothing  ia  uncertain  (Rian  n’est  sAr  qua 
la  Chose  incertaina).  It  ia  batter  to  begin  with  doubt  and  and  in  certainty, 
than  to  begin  with  certainty  and  and  in  doubt.  Distrust  and  doubt  is  tha  wise 
strategy  of  tha  novice. 

Brunt  one  who  has  proved  it.  Virgil,  Aanaid,  70-19BC 

1.  T.B. ,  BXXSXMG,  J.M.  «  BBBi "AOBWB,  H.6.  (1987)  Workload  and 
Situation  Awareness  In  future  Aircraft.  In;  Recant  Advances  in  Cockpit  Aids 
for  Military  Operations.  London;  RAsS. 

2.  DHOMMMDQO,  a.  ( 1982)  Attention  and  Awareness  in  Cognitive  sad  Motor  Skills. 
In;  Aspects  of  Consciousness,  Vol.  3.  London;  Academic  preaa. 

3.  SMLCOB,  s.J.  (19MS)  An  Information  Proceaeing  Approach  to  Decision  Support 
in  tha  Cockpit;  Help  or  Hindrance?  Proceedings  of  the  Workshop  on  the  Human 
Electronic  Crew,  Xngolatadt  (In  Prase). 
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data  (runway  length,  etc.). 
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due  ib  nmncrona  phyaical  and  paychological  factoo. 

Aa  described  above,  the  PVI  Manager  can  help  to  alleviate  theae  problema  by  giving  the  pilot  high 


Isvei  control  over  the  contents  of  the  display  suite  sad  individual  formats  through  the  selection  of 
operating  modes  and  um  of  soeemm  commands  to  manage  the  display  suite.  A*  importantly,  the 
puot  thQ»»Vf  be  able  to  control  the  complexity  of  individual  display  foesuus  by  directing  the  FVT 
Mmmw  ^  jtitefligently  tteelnoer  the  display.  For  instance,  the  pilot  may  first  want  to  tee  all  aircraft 
vidua  t  «yea  am  to  help  Itim  oaks  mission  planning  decisions  and  than  aak  the  PVI  Manager  to 
declmat  tbs  display  to  show  only  aacrift  flat  are  of  immediaSB  tactical  importance.  Several  levels 
of  dsdaaadng  may  be  poariMs  and  dadaaating  may  pertain  to  the  type  or  aiicmft  shown,  attributes 
of  tfaeee  aircraft,  tup  data,  navigation  data,  aensor  data,  and  other  information  that  sight  be  on  an 
integrated  display. 

The  intelligent  dachusedng  of  displays  is  an  extension  of  display  task  tailoring  logic  that  gives  the 
pilot  an  easily  accessible  and  lateral  ananUon  of  control  over  display  content  It  baba  him  deal 
qnkto  with  tactical  atoationswhidi  ate  inhatenttycoaaniex  or  IrigMysttessftiL  At  the  same  time, 
h—  «  rimflie  mii^iii—io—i  iw«mi  *«  wm — detailed  when  needed  To  be 

aflbctivs!hcm«vw,  deohttiwing  mast  be  bean waephiatiemad logic.  Ititnotsufflcieat,  for 
siamiile.  at  eliarinasc asm  aircraft  nefilde  nf  a  certain  raimi  ofownshto tinea in  timimt  — hat 
an  ahoaft  tens  of  miles  sway  can  pose  a  Ugh  threat  The  objective  is  to  faeflfcam  the  pBofs  ability 
to  abstract  Ugh  value  mfoanaston  As  a  mealt,  complex  talas  ate  needed  to  determine  which 
information  is  of  high  value  in  a  given  situation. 

4.  HBGHUGHTOUTI1CAL  INFORMATION 

In  some  cases,  it  is  not  sufficient  to  timpfy ^pro^tofonnation  to^tfaepOoc.  lUthcr,  foe  ayionic 


location  coding  (e^  master  caution  d 
symbols;  flashing  symbols;  tones;  and 


);  changes  m  color,  sixe,  and  shape  coding  of 


threats  (e^. 


rm*  fc—  m  .humlmewtet  m  mrft  —  hanrflrial  efto  in  that  Itiev 
wily  •fiatnrt  iht  |rVrt  n—*rmA  fa.  rriOni  These 

•  is  Uns  post  becanae  camion,  alerts,  and  warnings  am  often  "hardwired"  into  a 
which  does  not  tala  the  overall  context  of  a  problem  into  consideration.  In  future 
llgent  1 1 bm ir  ijinrnn  trill  lute  sun  mnn  Infir— Tf--  ■*- ngmgt— * 

nement  systems,  far  eaampfa.csndesnrtsithrin,  but  important  changes  in  the  status  of 


t  of  those  action  by  the  pilot. 


information  useful  So  his  task.  Lflawhhfoehmmn^  femmi^^^leggonic 


method  for  MghHghriag 


of  which  information  to  highlight;  and  selection  of  an  appropriate 


Jl-t - ■- 

fiaaemtf  » 


e  time  constraints.  A«  inch,  highlighting  sachnsqaes  need  schemed 

pilofs  concentration  is  not  disreptnltetite  point  thstdscisioM  aretxx 

.  To  prevent  this  problem,  Ughngfatiug  staagy  shoold  be  subject  to 
When  the  pilot  hessUecseddectotered  levels  of  information 
restrictive  rules  should  ba  applied  to  the  use  of  highlighting 


5.  INTERPRET  PILOT  COMMANDS 


When  interacting  with  intelligent  electronic  crew  systems,  the  pilot  needs  to  wort  at  a  "command" 
level  lather  than  ass  controller  of  individual  systems.  The  PVI  Manager  can  support  the  attainment 
of  this  goal  by  interpreting  the  intent  of  high  level  pilot  commands  and  directing  other  systems  to 
take  actions  needed  to  implement  that  intent  Foreutnple,ifa  pilot  is  assignedatagglt  via  data  link 
from  an  airbera  command  canter,  ha  may  accept  the  assignment  through  the  command  "Wileo 
assignment*.  The  PVI  Manager,  using  its  stared  knowledge  of  the  particulars  of  tbetanet  to  be 
snacked,  can  (a)  formulate  a  data  link  message  to  the  command  center  to  inform  them  or  compliance 
and  sand  it  to  the  communication  systems  tooa  transmitted,  (b)  foamlate  a  data  Uric  message  to 
wingmen  to  inform  them  of  new  tactical  objactlves,(c)  send  target  priorization  information  « the 
dmlin  assessment  system,  (d)  cue  tfaa  pi— ittwg  — h  seneoe  management  systems  to  <— **»«»»■ 

actions,  and  (a)  restructure  mah^arpoae  displays  ad  mttftnetion  controls  to  sappott  the  next 
phm  t+thm  engagement.  For  armther  example,  the  pilot  mqrdeflnaavolinne  of  egapace  that  he 
waits  searched  The  PVI  Manager,  working  fat  conjunction  with  the  Sensor  Manager,  can  define 
the  min  of  senaosa  to  be  employed  in  the  teach,  the  modes  of  each  sensor  to  be  usodl  and  the 
priority  to  be  given  to  the  aeareh  task  given  concurrent  tasking  of  the  sensor  systems. 


_ _ _  _  lamhority  thapflot  wants  to  give  the 

electronic  crew  systems.  He  may  choose  commends  that  provide  additional  direction  or  otherwise 
ronstnrinthewtponseof  the  electronic  crew  systems.  The  joint  lobe  made  hew  is  that  the  pilot 
needs  access  toasuccinct,  high  level  command  language.  This  maltimraiia  language  win  include 
control  inputs  made  through  dedicaied  switches  on  the  stick  and  throttle,  menus  and  virtual  switches 
on  display  formats,  and  voice  recognition  lysteim.  Indeed,  the  pQot  may  make  the  same  command 
using  different  media  risuanrtfng  on  which  control  actions  aw  more  coorenjent  in  a  given  situation 
ThePVir  *  -  •• 


language  fnr  tranimissiun  to  other  elactronic  systems.  Additionally,  the  pilot  may  uaa  mixed  media 
far  a  command  (0Lg*  tench  a  target  on  a  ecaean  and  uaa  a  voice  command  to  ted  the  syatem  whet 
action  to  ate  regarding  that  target).  In  tide  erne,  the  PVI  Manager  needs  to  asaodam  related  control 
inputs. 


5.  DISCUSSION 


The  PVI  Manager  works  to  ensure  that  the  pilot  has  the  right  information  in  the  right  format  the 
ngnx  am  ao  ms  no  is  not  avnewa  or  srewmon.  luinm,  n  cniDics  me  puot  k>  cumroi  toe 


A  prototype  PVI  Maeagwhae  been  developed  as  part  of  Boeing  research  on  avionic  expert  systems 
(IX  ft— yt«y  immisjimm  milts  a hltnhoanl  — t»ft—— » »h»g m— gt— t 
tasks.  Sfama  no  ”eapans*  exist  for  PVI  Mawgamanttaaki  an  ftmmabcraft,  new  procedures  had  to 
be  created  for  dtvelrafag  dm  haowledgabwe  for  dm  pweotype.  Toi«ttwhh,wefi-eiabllihed 

crow  systems  analysis  pm  oedweawwa  read  to  define  mierina  fuquiremanti,  pilot  and  system 
nincngw,  mgpMoimoBwwmigaoimpgCTwat  imnoim^ini »««  nvnwii  o» 
reeevch  on  advaanad  pictorial  foeaad  dkptaye  (2),  was  used  to  oontmtet  a  prototype  PVI  ad  PVI 
Manager.  Rapid  peorotypingtechaiqBaa  lead  in  expert  ij  item  Jandoonwre  irere  mea  employed  to 

IP1I»  rYI  QMHS  CCx'33fpCi  m  BlrVl  AOM|k  UUWKUfl  mm  mm 

ie  search  so  avreests  the  FVI  Manager  operating  in  conjunction  with  other  expert  systems  used  for 
situation  aseatamanf;  twlseiwi  planning,  and  oner  advanced  avionic  fanedosm 
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PVI  Highlights  Important  Information 


Interpretation  of  Pilot  Commands 


of  Expert  System  Technology 


f ' 

( 


T 


Conclusions 


\  i  r. .  *,* 


in 


r 


- OUto.  tHAM 

AVIOH8  MARCEL  DASSAULT  -  BRSOUET  AVIATION 
7S,  qua!  Marcel  DASSAULT  •  92214  SAINT  CLOUO  -  PRANCE 
August  29.  IBM 


The  purpoM  of  this  paper  is  to  press nt  tba  concept  of  the  'Electronic  Copilot*  now  In  e  feasibility  stage 
at  0ASSAULT-8REQUET  under  DRET  contract  *8-34-407 


It  first  illustrate  the  state  of  the  art  in  aircraft  application*  of  A. I.  technology  at  DASSAULT-BREGUET. 
Then  a  general  presentation  of  our  works  relative  to  the  'Electronic  Copilot'  shows  the  importance  of  Ihe 
Man  Machine  Interface.  The  techniques  Invest Ignled  by  0ASSAULT-BRE8UET  with  various  partners,  in 
order  to  tackle  the  problem  are  reviewed. 


In  the  A.I.  domain  some  interesting  research  are: 

•  Uncertain  and  temporal  information  processing. 

•  Real  time  inferring  for  on-board  A.I.  systems. 

•  Cognitive  modeling  of  the  pilot... 

Concerning  M.M.I.  problems  some  new  Ideas  are: 

•  Pertinent  announcement  of  cautions  and  warnings. 

•  Synthetic  data  presentation,  especially  terrain  data  files. 

•  Workload  assessment  experiments, 

•  Voice  Interactive  devices  Integration. 

•  Stereoscopic  presentation... 

A  perspective  on  a  future  system  integrating  these  techniques  Iter  the  benefit  of  the  pilot  during  a  low 
altitude  Ingress  mission  Hlustrate  the  concept. 


Artificial  Intelligence  at  DASSAULT-BREOUET 

Since  1981,  the  Artificial  Intelligence  teams  of  DASSAULT-BREGUET  have  been  implementing  various 
systems  for  aircraft  applications. 

Some  of  the  major  developments  are: 

•  In  the  context  of  air  combat  simulation 

Air  combat  has  been  the  subject  of  many  studies  at  DASSAULT-BREGUET.  An  expert  system  has 
been  developed  to  simulate  dogfights  (CHAMPIGNEUX  *5)  This  expert  system  is  capable  of  rea¬ 
soning  bom  the  tactics  and  strategies  acquired  from  Ihe  specialists'  experience  in  order  to  By  a 
combat  aircraft  against  a  single  opponent. 

For  multiple  aircraft  engagement  It  has  proven  necessary  lo  rreale  expertise  without  collecting 
rules  from  human  experts.  Automatic  learning  appeared  to  be  the  answer  to  this  problem.  To 
demonstrate  the  feesibllity  of  this  approach  and  study  the  methodology  required  for  Implementing 
these  techniques,  experiments  on  a  real  application  were  initiated  in  1985,  and  a  software  environ¬ 
ment  to  perfect  the  expertise  in  multi-aircraft  combat  tactics  by  automatic  teaming  has  been  created 
(GILLESM). 

•  In  the  CAD/CAM  context: 

An  environment  to  aid  a  mechanical  designer  has  been  studied.  II  allows  the  designer  to  modify  his 
mechanism  and  to  verify  Its  consistency  easily  and  quickly.  A  knowledge-based  system  analyses  the 
mechanism  in  order  to  deduce  the  functional  dimensions  and  a  reassembly  algorithm.  A  prototype 
of  this  system  hes  been  successfully  tested  on  several  aircraft  mechanisms  (HUTT  M). 

•  In  the  security  analysis  context: 

There  is  a  strong  need  for  failure  analysis  in  the  aerospace  industry.  The  fault  tree  method  is  often 
used  to  demonstrate  the  reliabilities  of  complex  systems.  The  design  of  such  a  tree  for  a  given 
system  Is  an  art  and  the  graphic  form  of  the  tree  express  the  expertise  of  it's  designer.  A  software 
package  was  developed  with  Artificial  Intelligence  techniques  such  as  functional  programming  and 
object  oriented  programming.  It  allows  Interactive  analysis  of  the  reliability  of  complex  aircraft 
systems  such  as  the  terrain  following  system  of  the  Mirage  2000N  (CHAMPIGNEUX  M). 
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•  In  various  domains: 

Wa  also  investigate  the  interest  of  Artificial  Intelligence  In  teclinlcal  diagnostic,  natural  language 
understanding,  software  specification... 


The  Pttot  Assistance  Context  at  DASSAULT-BREGUET 

Conducting  penetration  missions  in  hostile  territory  has  always  raised  problems  of  workload  on  a  single 
pHot.  regardless  of  the  aircraft  configuration  considered.  These  problems  have  generally  been  solved 
by  applying  strict  mission  cbntroi  rides  or  by  adding  a  second  crew  member.  However,  in  a  single  sealer 
aircraft,  even  if  the  pilot  is  relieved  of  routine  and  repetitive  tasks,  mission  control  can  be  unecceptably 
complicated. 

Considering  that  Artificial  Intelligence  could  provide  answers  to  these  problems.  DASSAULT-BREGUET 
is  working  on  a  feasibility  study  of  this  approach  for  the  aircraft  of  the  1990-2000  decade  and  the  initiation 
of  the  necessary  research.  This  feasibility  study  Is  being  carried  out  by  DASSAULT-BREGUET  under 
DRET  contract  M-34-407  d  'Electronic  Copilot* 

The  'Electronic  Copilot*  corresponds  to  a  vast  protect  which  Investigates  the  cognitive  aspects  related 
to  analysis  of  the  various  areas  of  expertise  associated  with  pilot  aid  and  the  computer  science  aspects 
relative  to  implementation  of  Artificial  Intelligence  techniques  and  languages  in  airborne  systems 

We  strongly  believe  that  the  Electronic  Copilot  will  increase  the  importance  of  the  man  machine  interface 
as  It  will  generate  e  real  dialogue  with  the  Pilot.  This  will  require  Artificial  Intelligence  techniques  not 
only  to  generate  displays  or  messages  but  to  manage  the  pertinence  of  information  depending  on  the 
mission  phases  as  well  as  on  the  history  of  the  Pilot  activity.  It  will  he  a  central  task  for  the  Copilot  to 
infer  continuously  the  PHot  activity,  and  to  exchange  with  him  suitable  synthetic  information  in  order  to 
assist  the  decision  process. 


The  Artificial  Intelligence  Approach 

Artificial  Intelligence  techniques  and  languages  can  thus  be  used  to  model  some  of  the  pilot's  reasoning 
processes  in  order  to  alleviate  his  workload  in  a  hostile  environment  or  help  him  in  complex,  laborious 
or  repetitive  operations,  by  transcribing  in  a  computer  the  expertise  and  experience  acquired  by  the 
operators,  the  airframe  manufacturer  and  the  equipment  manufacturers. 


Unfortunately  the  present  state  of  Artificial  InteIHgenee  techniques  do  not  permit  a  direct  Implementation 
of  the  Electronic  Copilot  concept,  and  new  research  seem  to  be  necessary. 


•  First  of  all  the  environment  of  a  military  aircraft  will  not  he  a  static  well  known  and  precise  one. 

In  collaboration  with  the  LIFIA,  we  are  carrying  a  research  in  order  to  manage  uncertain  and 
temporal  information.  Our  approach  is  more  heuristic  than  the  fuzzy  logic  approach  and  we  try  to 
taka  advantage  of  the  operational  expertise.  This  will  allow  reasoning  about  the  certainty  of  infor¬ 
mation.  the  quality  and  interest  of  hypothetical  worlds,  and  the  confirmation  of  information  during  the 
mission. 

The  Artificial  Intelligence  problems  underneath  are  concerning  Hie  combinatorial  explosion  of  hy¬ 
pothesis,  the  modellsation  of  time  and  uncertainty,  the  connection  of  real  world  models  with  empir¬ 
ical  expertise  of  military  missions. 

•  We  also  decided  to  evaluate  the  real-time  performance  of  Artificial  Intelligence  mechanisms  as  re¬ 
gards  the  constraints  Inherent  in  the  suitability  of  an  expert  system  for  airborne  use. 

Any  system  operating  in  real  time  is  faced  with  two  types  of  constraints: 

■  The  constraints  specific  to  the  data  to  be  processed:  the  Information  is  generally  heterogeneous, 
sometimes  incomplete,  unsure,  even  contradictory  and  in  all  cases,  varies  with  time  To  this 
intrinsic  nature  of  the  data  are  also  added  the  constraints  of  exchanges  by  messages  which  are 
synchronous  or  asynchronous,  random  and  especially.  Independent  of  the  processing. 
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•  The  response  times  allocated  ami  tlw  times  allocated  for  execution  of  the  processing:  a  real* 
time  system  must  be  able  to  cope  with  peak  computation  workfoada.  manage  multiple  leaks, 
conflicts  and  Interrupts.  Alter  data,  ate  while  complying  with  the  time  Hmtta  set.  This  necessary 
control  of  execution  therefore  requires  high  processing  power,  processing  managed  by  tasks 
with  different  priorities.  Interrupt  levels  in  the  processing,  a  task  scheduler,  etc. 

However,  an  Expert  System  is  poorly  prepared  to  accept  such  constraints: 

■  The  information  processed  Is  more  often  symbolic  than  numerical,  the  data  generally  have  a 
durable  or  even  Axed  value,  which  means  thai  In  most  cases  there  are  problems  of  nonmonot¬ 
ony. 

■  It  is  difltcult  to  predict  Am  performance.  In  particular  as  regards  the  execution  times  of  the  rea¬ 
soning  process,  as  they  depend  on  the  strategies  used,  the  order  in  which  the  information  is 
processed,  etc. 


In  addition  to  these  constraints,  there  is  also  the  problem  of  memory  volume:  At  applications  are 
generally  'greedy'  in  memory  requirements  and  all  use  a  'garbage  collector*  for  memory  manage¬ 
ment.  This  results  in: 

•  A  cost  in  memory  recovery  lime 

■  The  requirement  for  dynamic  memory  management 

•  Difficulties  in  interfacing  with  algorithmic  languages,  as  the  data  do  not  have  a  fixed  allocated 
location 

•  Problems  of  completeness  If  the  memory  size  Is  limited 

■  Finally,  an  unpredictable  execution  time. 

In  the  area  of  airborne  applications,  the  real-time  constraints  mentioned  are  crucial  and 
DASSAUIT-BREGUET  in  collaboration  with  E3D  is  studying  these  problems  in  the  domain  of  system 
status  evaluation. 

Cognitive  modelling 

Our  concept  of  the  Electronic  Copilot  is  dearly  puffing  the  Pilot  in  the  loop.  The  Electronic  Copilot 
will  only  propose  decisions  to  the  pilot  or  present  Information  pertinent  for  the  Pilot  decision  process. 
The  Pilot  will  be  free  to  accept  or  not  the  proposition  and  no  automatic  decision  will  be  taken. 

This  implies  that  pertinent  Information  Should  be  managed  in  order  to  minimise  the  divergence  with 
the  Pilot  line  of  reasoning.  For  instance  a  particularly  Important  point  in  pilot  aid  is  Altering  of  the 
alarms  and  management  of  emergency  procedures.  The  pilot  must  be  able  to  supervise  control  of 
the  various  aircraft  systems  in  all  situations,  including  failure  situations. 

The  aid  in  understanding  failures  and  managing  emergency  procedures,  requires  analysing  the  ef¬ 
fect  of  a  failure  to  inform  the  pilot,  recommend  actions  lo  limit  the  impact  of  the  failure  and  alert  Ihe 
pilot  to  degrading  of  the  Alght  envelope  of  the  aircraft 

A  characteristic  example  will  give  an  idea  of  the  difficulty  involved  in  this  problem:  A  failure  of  the 
brake  system  detected  at  M  1.8/30.000  A  must  not  he  handled  In  the  same  way  as  such  a  failure 
occurring  during  approach  with  the  landing  gear  extended  In  the  first  case  the  failure  must  be  in¬ 
dicated  because  it  may  affect  the  choice  of  recovery  base  or  diversion  base,  but  the  pilot  does  not 
have  to  make  a  snap  decision.  In  the  second  case.  Ihe  failure  must  he  reported  to  allow  a  decision 
to  be  made  on  whether  or  not  to  abort  the  approach  in  progress.  The  decision-making  itself  depends 
on  the  available  runway  length  and  state,  the  remaining  fuel  quantity,  etc. 

Such  a  problem  requires  taking  the  pilot  workload  into  account,  because  decisions  must  be  pro¬ 
posed  with  acceptable  response  times  for  the  pilot.  This  means  a  real  management  of  pertinent 
information  to  the  Pilot  according  to  a  cognitive  model  of  the  Pilot  during  rapid  process  control. 

A  research  action  has  been  Initiated  In  this  domain  with  the  CERMA,  based  on  the  psychological 
concepts  of  plans  scripts  and  schemes. 
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Man  Machlna  Intarfaea  dwlgn  and  Artificial  Intelligence 

More  and  more  data  are  avaHabte  in  the  core  of  the  different  computers  lilted  in  modem  aircrall.  In  many 
Balds  this  important  amount  of  data  is  net  ready  to  be  directly  displayed  to  the  crew  (terrain  liles.  e.g  ). 
Anyway  crew  don't  like  to  be  given  Important  amount  of  data  :  they  only  wish  to  get  the  information  they 
need  at  the  time  they  need.  That  Is  the  reason  why  our  Company  is  working  for  many  years  to  find  best 
appropriate  ways  lo  display  information,  taking  in  consideration  that  ’a  picture  Is  stilt  worth  a  thousand 
wards'.  This  work  has  led  DASSAULT  Company  for  many  years  to  develop  In  lighters  head-up  flying 
using  velocity  vector  and  energy  rate  far  all  the  mission  phases  and  adding  very  powerful  high  order 
guidance  symbols  in  many  other  modes  :  for  example  the  synthetic  runway  with  associated  guidance 
box. 

We  are  now  thinking  that  Al  techniques  can  be  very  helpful  in  this  quest  for  best  interfaces  (LARROQUE 
87).  The  idea  is  to  continuously  adapt  the  display  contents  to  the  situation  and  to  the  pilot  preoccup¬ 
ations.  Moreover  Al  driven  cockpits  are  foreseen  as  natural  extensions  of  our  present  know-how. 

The  importance  of  man-machine  interface  design  has  led  our  Company  to  make  an  always  increasing 
use  of  simulation  techniques.  Several  tools  are  used  to  define  the  cockpit  and  all  the  software-driven 
interfaces.  Final  assessments  and  adjustments  are  made  with  an  important  participation  of  flight-test 
teams  in  our  simulation  facility  namad  OASIS  (for  Outit  d'Aide  A  la  Specification  des  Interfaces  SystAmes 
i.e.  Man-Machine  Interface  Design  Tool)  located  in  Istres. 

OASIS  has  been  used  mainly  to  develop  the  different  versions  of  the  MIRAGE  2000  and  to  design  EFIS 
fitted  in  many  FALCON.  Many  works  have  also  been  made  on  OASIS  for  the  RAFALE  Demonstrator  and 
we  are  now  in  progress  to  define  the  new  ACE  RAFALE-D  crewstation.  We  also  have  an  important  ac¬ 
tivity  of  research  in  displays  and  controls  tor  very  low  level  penetration.  MLS-landing.  air-to-air  multiple 
engagement... 

in  order  to  know  which  are  the  best  directions  in  our  cockpit  designs  we  sometimes  use  'workload  as¬ 
sessment'  methods.  It  was  the  case  with  studies  concerning  on-board  use  of  voice  processing  and  its 
relationship  with  other  means  of  dialogue  :  displays,  keyboards,  dedicated  or  soft-keys...  (BUSTAMANTE 
M) 

The  'Electronic  Copilot'  study  is  now  trying  to  merge  our  knowledges  in  both  fields  Al  and  MMI.  Some 
directions  appear  as  very  promising  in  order  to  simplify  from  the  pilot  point  of  view  the  use  of  all  the 
aircraft  functions.  We  wlH  take  hereafter  an  example  lo  Illustrate  how  pragmatic  is  our  approach  due  to 
our  strong  willing  (and  need)  of  real  future  on-board  applications. 

But  this  slate  of  mind  doesn't  exclude  other  axis  for  our  research.  For  example,  we  have  also  planned 
with  SOQITEC,  one  of  our  subsidiary,  to  study  stereoscopic  presentation  The  aim  is  to  use  another  na¬ 
tural  channel  of  the  human  perception.  Experiments  are  foreseen  in  this  context  using  devices  already 
developed  by  ETCA  (Eteblissement  Technique  Central  de  I  Armament  A  Bagneux)  in  a  simulator  featuring 
a  new  fighter  cockpit. 


ALARM  FILTERING  :  AN  EXAMPLE  OF  Al  APPLICATION 

In  conventional  aircraft,  failuras  and  major  events  concerning  all  the  system  are  usually  presented  by 
amber  and  red  lamps.  Most  of  the  time  the  lighting  of  tamps  also  initiatn  warning  tones.  This  kind  or 
device  has  many  advantages  especially  simplicity,  reliability,  independence  ...  but  from  an  MMI  point  of 
view  it  presents  draw-backs  as  : 

•  a  systematic  behaviour  who  doesn't  care  about  conditions  in  which  the  event  is  happening  For 
example.  'AC  generator  failure'  can  flash  if  the  generator  fails  or  if  the  engine  fails  or  even  if  the 
engine  is  stopped.  As  a  consequence  crew  are  used  to  fly  In  some  conditions  (air-show  presenta¬ 
tions.  e.g.)  with  systematic  alarms,  they  switch  out  the  audio  warning  and  of  course  their  ability  to 
perceive  new  signals  and  read  in  case  of  a  'true  failure'  is  reduced.  Another  consequence  is  the 
resulting  ‘Christmas  tree*  in  major  failures  where  the  faulty  element  leads  events  in  succession. 
Many  squares  are  flashing  ;  among  them  you  have  to  find  the  guilty  (and  the  fighting  order  is  not 
always  a  good  Indication  !). 

•  the  crew  has  to  divert  his  attention  from  the  main  task  In  order  to  know  which  is  the  displayed  alarm 
even  If  they  don't  have  to  read  within  a  few  seconds 

In  the  new  generation  a/c  technology  improvements  make  possible  data  colleding  and  processing  about 
big  mass  of  parameters  even  those  concerning  engine,  hydraulics  devices,  brakes,  fuel  Our  Company 
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ie  already  applying  thee*  malhoda  for  many  yeara  on  the  different  version*  of  the  MIRAGE  2000.  On  the 
RAFALE  we  have  taken  advantage  of  this  know-how  lo  re-organise  the  ceutlon  and  warning  announce¬ 
ment.  First  steps  have  been  carried  out  on  the  RAFALE  demonstrator  and  we  are  working  now  on  the 
pertinence  of  the  announcement  Itself. 

On  the  RAFALE  demonstrator  we  have  experimented  a  new  way  (or  the  presentation  by  the  association 
of  a  speech  synthesis  device  and  dedicated  warnings  in  all  the  CRTs.  The  idea  was  mainly  to  give  the 
pitot  the  announcement  : 

•  at  the  best  place  :  the  alarm  has  to  catch  the  pilot's  attention  ;  this  is  obtained  both  by  the  audio 
warning  and  the  fact  that  written  messages  are  given  at  the  same  time  in  the  Head-Up.  the  Head- 
Level  and  the  Head-Down  displays. 

•  dearly  :  with  the  spoken  message  and/or  the  written  message  the  pitot  knows  at  once  the  impor¬ 
tance  of  the  event,  the  concerned  system  and  In  the  worse  cases  the  recommended  actuation  or 
manceuver. 

To  reach  these  aims  we  have  decided  to  manage  all  the  alarms  in  computers.  The  processing  slso  in¬ 
corporates  some  niters  taking  into  account  conditions  on  engine  status,  position  of  the  a/c  in  the  night 
envelope,  height  above  the  ground,  airspeed  or  ground-speed...  The  computers  are  also  able  to  display 
low  priority  alerts  only  if  no  high  priority  event  is  detected. 

Using  data  and  expertise  collected  during  more  than  300  night  hours  on  RAFALE  demonstrator  we  are 
now  working  with  Electronique  Serge  Dassault  under  contract  from  French  Government  in  order  to  im¬ 
prove  the  Altering  process  Al  techniques  seemed  to  us  very  well  adapted  to  treat  this  point.  The  most 
interesting  cases  are  concerning  the  behaviour  of  the  system  when  for  example,  an  unexpected  engine 
name  out  can  generate  a  lot  of  detected  misfunctlons  in  hydraulics,  air  conditioning,  electricity...  A  sec¬ 
ondary  goal  is  to  examine  if  some  false  alarms  can  be  avoided  by  comparison  of  non-independent  data. 


CONCLUSION 

Our  belief  is  that  Artificial  Intelligence  and  Man  Machine  Interface  Design  will  be  strongly  linked  in  the 
future.  This  seems  to  be  a  necessary  step  in  order  to  allow  efficient  management  of  complex  missions 
the  Pilot  will  have  to  perform.  The  Electronic  Copilot  will  be  a  good  assistant  for  the  Pilot  if  it  can  follow 
the  Pilot  line  of  reasoning.  This  means  that  the  Relevant  Information  Management  will  be  an  essential 
task  for  the  Electronic  Copilot,  and  that  Man  Machine  Interface  Design  should  be  considered  as  a  Reid 
of  operational  expertise  as  well  as  tactical  management  or  avionic  system  management.  We  are  now 
working  In  this  domain  in  order  to  allow  Human-Electronic  crew.  So.  Important  efforts  of  DASSAULT 
company  in  this  Reid  are  essential  for  the  design  of  best  competitive  fighters. 
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Major  Ronald  L.  SmII,  Captain  Carl  S.  Liras  and  Captain  John  P.  Seayuh 
Air  Pore*  Wright  Aaranaatieal  Laboratories  (AlWAL/PZaR) 
Wrlght-Pattersoa  APB,  OH  43433-4593  USA 


Zatsadoetioa 

Tho  continuing  evolution  of  fight ar  aircraft  technologies  ia  geoer- 
ating  aortal  wap  on  systme  that  an  fy tatoc,  haw  groator  rang*,  and  ara 
nose  lethal  than  a  vat  bafosn.  ttapcaaadantad  quantities  of  infooaatioo 
will  ba  available  to  futara  fighter  pilot#.  Poop ana  and  survival  ia  tha 
futura  air  conbet  arana  will  d^and  open  tha  pilot'a  ability  to  rapidly 
aaaiadlata  Chi#  voluaw  of  iafoenstion  into  an  aocurata  nantal  ioaga  of  tho 
aortal  situation  and  to  naka  tioa-oxitioal  wlsslbn- related  daoiaioo#  baaad 
on  that  aaaaaaaoat .  Za  orda#  to  explore  potantial  Artificial  Intelligence 
(AZ)  application*  daatgnad  to  holp  tbs  pilot  oopa  with  tfcia  aaaa  influx  of 
data  and  raapood  ia  a  knowladgaablo  and  tloaly  naanar,  tho  Defence 
Advanced  Raaaaroh  Projaeta  Agaaey  and  tba  3dr  Poroa  Wright  Aeronautical 
Laboratories  ara  aponaoring  tha  Pilot'a  Associate  prograo.  Thia  eeatract- 
ad  raaaatoh  affort  haa  two  prina  contractor*,  Lockheed  Aeronautical 
ffyatan*  Company  and  McDonnell  Aircraft  Ccepany.  the  Pilot'*  Aaaooiata 
prograo  paowidaa  an  applloation  aarlrnwaant  for  AZ  and  advanced  prooaaaor 
technologies  to  develop  an  "electronic  ccaiaaaabar*  for  a  poet-1999  aingla- 
•eat  fighter  aircraft. 

Thia  papar  daaoriboa  tha  functional  nodules  of  tho  Pilot'a  Aaaooiata 
and  entrant  dovolopi «t  statu*;  it  postulate*  technologies  required  to 
aeko  tho  Pilot'a  Aaaooiata  folly  functional;  and  it  annolada*  by  asking 
guest  tons  pertaining  to  tbo  long-tom  future  of  aloetrooio  ormooahor 
technologies. 

Pilot  * •  Aooooioto  — duloo 

The  Pilot's  Associate  design  aaploya  a  set  of  ein  cooperating  eapert 
syatanc  to  fom  a  daoiaioo  support  systao  for  futara  fighter  pilots.  Tha 
sin  expert  systseo  am  Mission  Planner,  Teotioa  Planner,  lituation  Asseas- 
osnt,  tystsn  Status,  Pilot-Wshiole  1st  erf  so*  and  Mission  Bnaeatlva.  These 
aspect  systsos  oust  cooperate  to  soooaaofnlly  perfoos  as  an  “electronic 
arevnaafeer,"  thereby  iaproviag  the  pilot'a  situational  awareness,  surviva¬ 
bility  and  eoobat  effectiveness  (figure  1) . 
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Mission  »lnm.  This  expert  system  caleulaCaa  tha  rout  a  based  on 
task  Information  aa  target  location,  foal  consumption,  timing,  thraat  con¬ 
ditions,  weather,  terrain,  pilot  preference  and  rolaa  of  engagmaant .  Rapid 
raapeoaa  of  the  Miaaion  Planner  to  nnarpsffifad  aiaaion  changer  iaeraaaaa 
tha  pilot' a  flsxlbtl tty  and,  therefore,  probability  for  successful  aiaaion 
accaapllshmant. 

Currently,  two  aaareh  tachnlqoaa  ara  being  evaluated  for  routa 
planning i  **  hauriatic  aaareh  (1)  and  dynamic  programming  (2) .  Baeauaa 
it  doaa  a  thorough  apaca  aaareh  for  tha  routa,  tha  Miaaion  PI anna r  ia  tha 
alowaat  executing  axpart .  But,  baeauaa  it  ueea  more  conventional  pro¬ 
gressing  techniques  aad  a  fairly  1  ini  tad  rule  bane,  it  may  bacoan  flight 
worthy  a  imply  by  nprlmltlag  tha  eonwantional  algorithm  ooaputatioaa  in 
militarised  hardware. 


Taatioa  PI  ameer.  She  Tactiea  Planner  ia  tha  expert  raaponaibla  for 
IQffltltlOV  letiOU  ^Midittt  thstltl  tACQIti*  xt 

Tactiea  Planner 'meat  alao  coordinate  tactica^for* mult i-airoraft  flight a, 
not  juat  single  aircraft  missions. 

CuzsiBt  Tsotios  Vlttaic  oipibilitlct  lato  ion  cooBdi* 

natad  two-ahlp  tactiea <  but,  only  ia  coaree  detail.  Per  example.  It  earn 
plan  beyond-viaual-ranga  air-to-air  attache  from  ha  ad-on,  forward  quarter 
or  bean.  She  toot  tea  Planner  enaurea  that  daalgnatad  targeta  are  aaalgned 
a  weapoe  ao  that  there  ia  aa  double  targeting  aad  eo  el, aged  targets.  In 
addition,  it  plana  reactions  to  unaipaotsd  ground  throats  or  target 

to  be  flight  worthy,  tha  Taetiea  Planner  needs  to  haws  quiokar 
reap  on  aa  a  plus  greater  flexibility.  If  it  rooaannads  only  a  few  tactiea 
ia  any  gleam  situation,  it  will  boeono  predictable*  aad,  in  air  ooabat 
predictable  means  vulnerable.  The  framework  exists  to  expand  tha 
kaowladga  base  to  yiald  a  debar  set  of  pooalbla  tactiea  for  considera¬ 
tion,  but  expanding  the  kaowladga  baas  will  make  tha  tactiea  Planner 
slower  net  quicker. 

tha  problems  of  flexibility  aad  raoponoluanaao  are  pervasive 
throughout  tha  Pilot's  Associate  ia  its  currant  daeal npnant  state.  Shat 
ia,  tha  Pllst’a  Aaaocl at#  needs  to  ha  able  to  reepowd  to  a  wider  variety 
of  situations  aad  give  qalohar  teapaoaaa  Co  these  situations.  1000000  of 
its  pervasiveness,  tha  isaaa  of  raapeoaa  tine  ia  dafarted  until  thn  "Wear- 
Tarn  Technology  Requirements”  section  of  thia  paper. 

Situation  Rasas  onset.  She  Iltuatioa  Aasssamant  module  la 

responsible  for  gathering  data  aboat  the  outside  world  (surface  and 
airborne)  by  correlating  aaeaor  data  into  fused  iafetaation.  It  priori¬ 
tises  threats  aad  f  spate  bared  upon  aiaaion  objectives,  location,  type, 
aad  estimated  threat  latent  lame. 

The  Situation  Assessor  seat  work  with  uaeartaln  os  incomplete  data, 
«dfc^,qy>  |y  ottrtfCBfcly  omi  of  tkt  aott  ohil Imyiiw  j^sobltii  ax 

rasas  where  (its  InpacS  ia  furthns  discussed la  tha  "Wear-Term  Technology 
nsijiil  1  smants"  saation  of  thia  papas) .  Wills  Situation  Aoaaaamsnt  assumes 

still  identify  thn  sen  lethal  threats  aed  eanltor  those  ease  closely. 
Postulating  high  lethality  throats  involves  identifying  or  ootinoting 
threat  typo,  oalnalePlng  goossotry  and  dot  graining  vulnerability  aad 
intent. 

Currently  tha  Situation  Aoosoonont  nodule  accounts  tor  uncertain 
data,  but  only  by  Spanning  worst  case  conditions.  It  awintalna  this 
onoertain  threat  object  data  ia  preparation  far  eventually  working  with 
Imperfect  sensors.  Tbs  Situation  Assessor  highlights  threats  that  exceed 
a  lethality  threshold,  aad  it  computes  missile  launch  regions  aa  part  of 
its  thraat  attribute  file. 


Iy*tw  Itieu .  This  wqpart  system  monitors  Cha  latarnal  aircraft 
subsystem  to  diagnose  and  suggest  corraetioaa  to  arror  conditiona  with 
tha  priaaxr  goal  of  determining  a  malfunction's  impact  on  tha  aiaaion. 
System  Statua  tracka  aircraft  capabilities  to  infoca  tha  plaanara  (Mia a ion 
Plrnmsr  and  Tactics  Planner)  what  limitations  to  account  for  in  thalr 
planning  activities.  Plana  that  ancaad  currant  ownahip  performance  ara 
filtarad  from  canal da ration  through  thia  interaction  batwaaa  Syatan  Statua 
and  tha  plaanara.  Syataa  Statua  must  alao  eventually  aoaitor  tha  health 
of  tha  Pilot's  Associate  ayatea  a Inca  it  too  ia  an  aircraft  aubaytaa 
subject  to  nalfunctiona . 

Currently,  Syataa  statua  can  diagnose  aany  ayataa  faults,  and  can 
correlate  nalfunctiona  to  detemiae  a  likely  eauaa.  Thia  eapert  auat 
aapand  its  knowledge  base  to  provide  full  coverage  of  aireraft  aubayataaa. 
Zt  canaot  rely  on  algoritbas  to  draw  tha  required  inferences  to  determine 
the  root  cause  of  nalfunctiona;  however,  it  can  use  subeysten  (e.g., 
engine,  hydraulic)  nodela  to  predict  failurea  baaed  on  trend  infometioa. 

Pilot -Vehicle  Interface.  The  role  of  the  Pilot-Vehicle  Interface 
nodule  ir  to  intelligently  process  the  infometioa  aval  labia  to  the  pilot 
in  order  to  el  Inina  to  adverse  workload  and  perfomanoe  lnpaet  a  due  to 
uncontrolled  infometioa  flow.  Pilot-Vehicle  Interface  reeeoaing  ia  baaed 
upon  the  current  situation,  relevant  information  content,  relative 
urgency,  end  pilot  intent  and  preferences.  Tha  level  of  detail  presented 
to  the  pilot  ia  controlled  to  provide  only  essmtial  information  daring 
tine  critical  mission  segments.  More  detailed  alienations  ara  available 
during  low  stress  situations. 

Currently,  the  Pilot-Vehicle  Interface  accounts  for  adoptive 
automation,  information  management,  end  coarse  pilot  intent  lnfereacing. 
adaptive  automation  functions,  preset  by  the  pilot,  direct  which  functions 
will  bo  perfomed  by  the  pilot  and  which  will  bn  automated.  Thia  pre¬ 
determination  helps  the  pilot  control  workload  for  e  given  situation  or 
set  of  functions.  Pilot  intent  lnfereacing  ia  the  most  difficult  Pilot - 
Vehicle  Interface  task  because  fighter  pilots  need  to  be  unpredictable  in 
oonbat.  A  bad  inference  could  moult  in  presenting  minaeded  information 
or  ranriTlng  needed  information,  than  causing  confusion.  So,  not  only  is 
intent  lnfereacing  difficult,  it  is  critical  to  user  acceptance  of  tbs 
Pilot'a  Associate,  in  situations  where  pilot  latent  ia  unclear  to  the 
Pilot'a  Associate,  infometioa  osn  cover  several  likely  possibilities . 
Current  intent  modelling  only  reasons  about  active  plans,  and  does  not 
change  displays  if  unsure  about  pilot  intentions. 

Information  mnagment  usee  inferred  pilot  intent,  mission  phase, 
pilot  preferences  and  an  astlnats  ef  pilot  cognitive  resources  to 
oonetruet  displays.  Information  ia  prsaantad  aurally  or  visually 
depending  on  thn  pilot's  task  loading.  Current  infometioa  management 
functions  provide  for  autenetla  display  changes  bsaad  on  Information 
requirements,  which  in  turn  am  driven  by  aiaaion  phase,  pilot  preferences 
and  notice  type  Information  (e.g. ,  aircraft  malfunction  descriptions) . 
Pilot  task  loading  ia  aatimtad  and  considered  for  node  of  presentation 
baaed  upon  modelling  the  pilot  as  a  set  of  processing  channels  i  visual, 
aural,  left  manual,  right  annuel  and  cognitive  (3) . 

Mission  Inactive.  This  eapert  ia  responsible  for  ensuring  the  smooth 
operation  of  the  Pilot'a  Aaaociata  syatan.  It  dona  thia  function  by 
tracking  and  updating  thn  mission's  progress,  plans,  goals  and 
constraints.  It  alao  establishes  priorities  for  systm  actions, 
renaanandetlons,  and  computational  resources ■  The  Mission  trecutive 
nadl area  disputes  between  tha  other  eapert  systems  so  that  thn  Pilot's 
Aaaociata  dona  not  present  conflicting  ceecamoodatioaa  to  the  pilot.  In 
addition  it  maintains  tha  mission  blackboard  as  tbs  massage  center  of  the 
Pilot's  Associate. 
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Weur-Tarm  Technology 

lb*  H  auat  operate  la  real-time,  that  la,  peasant  accurate 
information  to  tha  pilot  when  It  la  nesdad  ao  that  tha  pilot  can 
tlaaly  daelalona,  taka  appropriate  aetloaa,  and  improve  oombat  effective- 
oaaa  and  survivability •  Thla  oaad  baa  several  iapil cat Iona  regarding  hard¬ 
ware  acehltaotura,  software  architecture,  timing  ooaaidaratloaa  aad  data 
validity,  besides  thaaa  U  technology  laanaa,  thara  ara  tha  avionics 
iaaaaa  of  aiaa  and  weight  raatrietlooa,  aad  uaiag  — baddad  aapart  systems 
in  Ada  on  allitariaad  hardaara. 

■ardwara .  Aa  important  factor  in  making  tha  Pilot 'a  Associate  flight 
worthy  la  determining  a  hardware  archltaetuxa  to  meat  aiaa  aad  weight 
reatrictiona  and  real-time  naada.  Currently,  tha  eomputar  hardware  uaed 
for  Pllot'a  Associate  filla  a  large  room.  Obviously  tha  gaaaxml  purpoae 
ayebollc  aad  numeric  computer!  aatall  tha  overhand  aaadad  in  a  development 
environment  that  would  act  be  needed  la  tha  actual  avioaioa  architecture. 
On  the  other  hand,  there  la  ao  confident  way  of  determining  (even  within 
aa  order  of  magnitude)  tha  number  aad  aiae  of  proceaaora  required  to  run  a 
completed  Pllot'a  haeoelate  becauaa  tha  reaearah  aad  development  1a  lean 
than  halfway  oomplata.  whatever  typa(a)  of  proeeaaora  are  eventually 
required,  a  parallel  or  diatxifautod  processing  architecture  auat  be  uaed 
to  evea  approach  real-time  operation. 

architecture  coaaldaxatloaa  addreaa  multi-grained  psrsllnliant 
coaraa,  fine  aad  a  combination  of  both.  Both  types  of  granularity  provide 
advantage*  to  thia  application!  fine  grain  parallaliam  for  rule  firing 
aad  aaaoutloa  will  provide  mawlmnn  firing  rata;  aad,  ooaixaa  grain 
parallaliam  aupporta  ob ject-oriaatad  aytamo  aad  aultipla  iadapaadaat 
planners.  Therefore  a  mined  parallel  architecture  ia  appropriate  to 
achieve  tha  goala  of  flexibility  (i.a.,  amay  pleas  under  consideration 
simultaneously) ,  aad  re appeal vena  a a  (1.*.,  aaay  rules  firing  each  second) . 

Software.  Determining  aa  appropriate  software  axohitaatuxe  is  a  more 
difficult  problem  dam  to  a  lack  of  eeaaaaraial  products  aad  research 
experieocea;  whereas  parallel  hardware  ia  available  (for  a  pries) . 
Operating  aystmaa,  languages  aad  tools  (such  as  m  aad  US)  are  not 
currently  parallel land,  feme  parallel  software  aids  are  ia  tha  early 
reaearah  phases;  MX  <4),  MXCB  (9),  aad  Agora  (()  are  still  being 
developed,  aad  any  not  ha  available  for  usa  on  the  Pilot's  Associate 
program.  Ivaa  If  they  ara  available,  there  will  not  be  many  engineers 
experienced  in  usiag  these  parallel  tools  on  such  a  large  project. 

Xt  ia  likely  that  USP,  pins  variants,  will  b*  tha  predominant 
Pilot's  Associate  programed ng  language;  hot,  Ada  will  likely  ha  required 
for  eabedded  airborne  applications.  Oaa  of  tha  tools.  Agora,  works  with 
tha  C  progressing  language;  ao  tha  transition  path  from  LISP  to  C  to  Ada 
94m  prosi«ing. 

A  key  software  "tool"  ia  rap  resenting  tha  Pilot's  Associate  in  a 
QBditltOPdf  mbqk,  TaOfTlfh#Xi 1 9 

plea  aad  goal  graph  approach  (7)  provides  the  Pilot's  Aasoolat*  expert 
systems  a  framework  with  which  to  coordinate  actions  aad  ana ource  usages. 
3b*  set  of  active  pleas  give*  each  nodule  a  set  of  "marching  orders”  aad 
achieving  tha  goala  ara  the  aeasure  of  success  for  the  "caapaiga." 

Plana  also  provide  tha  saareh  space  for  possible  Pilot's  Assoc 1st* 
action*  aad  axplaaatioas .  This  planning  framework  Is  particularly 
lsgortaat  far  the  Pilot-Vehicle  interface  module  which  must  uadsxstaad 
pilot  eotioao  ia  order  to  infer  intent  aad  explain  Pilot's  Associate 
rscomsoodatioos. 


Timing.  It  Is  important  ia  tha  Pilot's  Associate's  anvironmant  to 
provide  good  answers  quiokly  —  the  optiaum  answer  does  ao  good  if  it 
arrives  late.  At  this  stag*  of  tho  program,  however,  neither  contractor 
team's  system  reasons  about  time,  except  ia  relation  to  aission  phases. 
That  is,  a  plaa  does  not  become  outdated  just  superseded  by  a  "better” 
plaa. 
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Anticipatory  planning  and  scheduling,  and  tradeoffs  in  accuracy 
varans  apaad  ara  only  baginning  to  gat  attantion  bacauaa  "sstisf icing" 

(tha  tarn  for  finding  a  "good  enough*  answer)  is  still  an  unresolved  AX 
research  problem.  ttaural  network  technology  has  demonstrated  promise  in 
addressing  satisficing  (8) ,  so  techniques  say  become  available  for  tha 
Pilot’s  Associate  to  find  a  good  answer  quickly. 

The  overall  issue  is  that  it  is  important  to  know  the  amount  of 
confutation  time  available  to  solve  a  problem.  Using  processor  time  and 
resources  to  solve  a  problem  without  having  some  idea  of  the  time  involved 
is  untenable.  The  high  speed,  high  threat  environment  of  future  jet 
fighters  does  not  permit  the  luxury  of  exhaustive  search  paradigms.  Hare, 
the  Pilot's  Associate  will  use  prior  knowledge  of  likely  situations  and 
time  constraints  to  prune  scam  answer  paths  before  expanding  resources 
making  useless  computations. 

Date  Validity.  Dealing  with  uncertain  data  is  another  vital  concern  of 
the  Pilot's  Associate.  The  isfact  of  imperfect  data  affects  the  different 
Pilot's  Associate  nodules  in  a  variety  of  ways.  The  Situation  Assessment 
and  System  Status  modules  must  account  for  conflicting  sensor  data  and 
false  alarms.  The  Tactics  and  Mission  Planners  must  account  for  worst 
case  and  stochastic  events  to  remain  flexible.  And,  the  Pilot-Vehicle 
Interface  must  account  for  possibly  erroneous  assumptions  of  pilot  intent 
to  prevent  presenting  useless  information  to  the  pilot. 

Truth  maintenance  and  reasoning  with  uncertainty  are  both  current  AI 
research  issues  of  great  import  to  many  applications.  Their  goal  is  to 
account  for  data  consistency  and  accuracy,  as  well  as  what  to  do  about 
backtracking  due  to  proven  false  assumptions.  Wills  progress  has  been 
made  —  in  part  because  of  the  technology  pull  from  the  Pilot's  Associate 
program  —  there  ara  still  many  research  issues  awaiting  resolution. 

Long  Texas 

The  current  Pilot's  Associate  research  and  devolnpmant  effort  will 
continue  until  1992.  At  that  time  the  Pilot's  Associate  is  expected  to 
run  in  real-time  in  a  full  mission,  piloted  simulator.  As  for  operation¬ 
ally  flying  expert  systems,  hardware  capabilities  will  probably  limit  tha 
USAT  to  portions  of  Pilot's  Associate  functionality  flying  on  advanced 
technology  aircraft,  for  example  the  Advanced  Taatical  fighter  or  Advanced 
Technology  Bomber,  in  the  add-  to  late- 1990's. 

Prior  to  full  operational  incorporation  of  Pilot's  Associate-like 
systems,  samller-acale  expert  systems  will  undoubtedly  fly  aboard  experi¬ 
mental  aircraft.  In  fact,  seme  cosyanlea  plan  to  translate  expert  systems 
developed  in  U3P  to  languages  supported  by  existing  avionics  architec¬ 
tures  for  flight  testa  in  the  next  few  years. 


Conclusion.  Tha  Pilot's  Associate  research  and  development  progrem  is 
a  major  step  toward  operational  application  of  electronic  crewmember 
technologies.  Carried  to  an  extreme,  future  programs  may  include  develop¬ 
ment  of  fully-autonomoua  decision-making  weapon  systems,  thereby  complete¬ 
ly  replacing  human  crewmembers .  This  davelopamnt  may  not  be  desirable, 
even  though  it  may  be  technically  feasible.  Determining  the  desirability 
of  replacing  humans  with  computers  in  weapon  systems  requires  addressing 
some  difficult  issues. 

Our  purpose  in  this  section  is  to  pose  soma  questions  in  need  of 
answers  before  replacing  human  pilots  with  autonomous  electronic  creumem- 
berst  Are  we  prepared  to  let  machines  make  decisions  to  intentionally 
kill  humans?  Are  we  willing  to  allow  a  machine  to  not  take  over  it  the 
human  operator  makes  a  fatal  error?  Nhat  about  cost?  The  cost  of  life 
support  equipment  for  chemical,  nuclear,  and  biological  warfare  may  make 
direct  human  control  prohibitive  for  some  missions;  can  we  afford  not  to 
replace  the  pilot? 
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SIMM  questions  highlight  some  of  Cha  laauM  to  bo  resolved  as  wo 
experiment  with  electronic  crewmember  technologies .  As  experiments  auoh 
aa  allot ' a  Associate  continue,  wo  ahould  bogia  to  answer  sosm  of  tho  aboro 
questions  to  determine  tho  limit a  to  which  wo  should  go. 
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EXPLANATION  OF  WORKSHOP  ACTIVITIES 
(Last  Two  0  a  y  s  ) 

WORKSHOP  TASKS 

The  keynote  address  set  the  overall  focus  for  the  meeting  by 
presenting  the  following  Issues: 

0  Is  the  pilot  always  in  control? 

0  How  nature  does  the  EC  have  to  be  to  be  useful? 

0  How  does  the  EC's  executive  program  function  effectively? 

0  What  level  of  security  clearance  should  the  EC  have? 

0  Will  the  pilot-EC  teaming  philosophies  be  the  sane  in  different 
countries? 

During  the  part  of  the  meeting  devoted  to  paper  presentations, 
these  issues  were  discussed  very  frequently,  and  when  it  was  time 
for  the  workshop  portion  of  the  meeting,  the  issues  formed  an  overall 
framework  to  guide  the  work  efforts.  However,  in  order  to  get  into  the 
issues  in  more  detail,  additional  structure  was  needed.  The  form  on  next 
page  X  provided  such  a  structure.  Both  of  the  key  factors  of  the  meeting 
(AI  and  the  Cockpit)  were  the  main  topics.  In  addition  for  each  of  these 
topics,  three  different  areas  (state  of  knowledge,  unresolved  issues, 
and  potential  directions)  were  considered  in  the  discussions. 

The  participants  vmre divided  into  six  multi-national  teams,  and 
each  team  utilized  the  same  form  to  structure  the  discussions.  After 
a  series  of  very  lively  interchanges,  each  team  came  up  with  its 
conclusions  for  each  of  the  six  cel!j  on  the  form.  The  team  chairs 
presented  their  conclusions  in  the  plenary  session  of  the  meeting.  The 
six  teams' results  are  given  in  Section  7. 
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THE  HUMAN-ELECTRONIC  CREW:  CAN  THEY  WORK  TOGETHER? 


workshop  OBJECTIVE.  To  identify  the  state  of  knowledge,  unresolved 
issues  and  potential  directions  in  aircraft  applications  of  Al  technology 
and  the  impact  on  the  cockpit  of  the  Human-Electronic  Crew. 

FORMAT  OF  WORKIN&  GROUP  ASSIGNMENTS 


AGENDA 

TOPIC  j 

Al  TECHNOLOGY 

COCKPIT  IMPLICATIONS 

1.  STATE  OF  KNOWLEDGE 

Levels  of  understanding. 

Current  practics  methods 
and  techniques. 

1.1 

9 

• 

2.1 

9 

■ 

2.  UNRESOLVED  ISSUES 

Areas  of  uncertainty. 

Resewch  and  devatopmant 
requirements. 

1.2 

9 

• 

2.2 

9 

• 

3-  POTENTIAL  DIRECTIONS 

Alternatives.  Choices. 
Prloiitlss 

Costs  /  beniffis. 

1.3 

9 

• 

2.3 

9 

• 

N.B.  All  groups  to  address  ail  ceils  in  the  order  indicated. 


sample  issues 

(1)  What  is  the  currant  stale  of  the  ait  needed  to  support  the  concept  of  the  Human/Esctronlc 
Crew? 

(2)  What  technical  areas  should  receive  the  most  emphasis  In  the  immediate  future? 

(3)  Whet  sort  of  schedule  for  operational  application  of  this  concept  are  the  experts  willing  to 
predict? 

(4) How  far  wfll  the  concept  be  pursued  l.e.  are  we  moving  along  a  path  toward  replacement  of 
the  humon  pilot? 

USEFUL  QUESTIONS 

PRIMARY-  What?  Which?  Why? 

SECONDARY-  How?  Who?  Where?  When? 

POTENTIAL  REQUIREMENTS  AND  CONSTRAINTS 
PRIMARY  -  Operational  (Environmental) 

Technical  (Physical.  Computational) 

Psychological  (Social,  Emotional,  Moral) 

SECONDARY  •  Economical.  Pofitical,  Physiological,  Biological,  Sociological,  Philosophical. 


140 


At  first  w*  ttKUMi  atawpaw  that  Arm  amemmm ary  to  progress 
from  automation  to  AX.  Ha  concluded  that  AX  i#  dmM  if  either  Ik* 
uncertainty  of  information  or  the  complex)  ty  of  aituation  attribute*  «CMd 
the  human' •  attention  apan.  Aa  french  approach  presented  ae  the  meeting 
of  fora  expert  ayatam  modal  aa  as  optional  "add-on*  capabilities.  tha  B>  and 
OX  approach  la  mora  toward*  aa  integrated  ayatam.  Tha  facet local  module*  w* 
idantlflad  aa  eaadldaeaa  for  hi  war*  aa  fo llama  t  Diagnostics  (probably  tha 
fir  at  to  b*  available),  Sanaor  Fusion,  In-flight  Hi  salon  Ae-g  leaning. 
Situation  Taxonomy  (a  vary  critical  aad  nacaaaary  modal*),  Sanaor  Wanapamant, 
Thraat/Combat  Management,  Intalligaaft  Cockpit  Data  Mamagsmant,  aad  finally 
the  Cxacutiva,  binding  the  other*  together  aad  mediating  between  than. 

1.2.  U  -  Omramolved  laamea 


There  ia  uncertainty  about  whether  or  not  we  can  really  aooceed  in 
building  a  fully  developed  real -time  SC.  Validation  aad  Verification  (v  &  V) 
procedure*  for  real-time  ayatana  praaant  critical  laeuee  which  will  detaxmin* 
whether  or  not  tha  MG  ooneapt  can  fully  aucoaed.  toother  critical  iaaua  ia 
whether  or  not  wo  ahould  allow  machine  learning?  Moat  they  lean?  Can  they 
learn?  Should  they  learn?  Ha  concluded  that  they  may  be  capable  of  learning 
but  they  must  acquire  new  rules  or  inference  capabilities  only  with  our 
approval  and  consent.  The  last  point  was  that  AI  developers  are  currently 
working  on  tractable  problem  domain*  where  progress  can  be  easily  made, 
however,  we  don't  know  yet  what  should  be  done  to  best  serve  operational 
needs. 


There  is  a  controversy.  Oo  w*  model  human  cognitive  processes  or  do  we 
go  just  for  results?  At  mi nlmnm  what  is  required  is  a  matrix  or  taxonomy  of 
situation  attributes,  reasoning  processes  and  decision  behaviour  attributes. 
Tha  issue  ia  what  can  be  achieved  by  pushing  autcamtion  to  the  maximum?  Do 
we  really  want  to  do  that?  Is  it  the  best  way  to  go  or  not? 


State  of 


There  is  a  need  to  ra-addreas  tha  requirements  for  cockpit  controls  and 
displays  resulting  from  tha  information  explosion  of  AX  system  module 
outputs.  Ha  can  expect  controversy  over  whether  this  should  be  an 
evolutionary  or  revolutionary  process.  He  decided  that  at  least  tha  hardware 
must  follow  an  evolutionary  process  of  development.  Solving  the  information 
management  problem  probably  requires  a  revolutionary  approach. 


Integration  is  the  kayl  Tha  modules  may  be  independently  developed 
because  of  the  different  personnel  and  capabilities  of  tha  Campania*  involved 
in  building  the  modules.  Tha  airoraft  integrators  must  be  capable  of 
explicitly  specifying  the  functions  and  outputs  oi  the  modules  before  sub¬ 
contracting  the  development  activity.  As  with  the  software,  deriving  the 
requirement  specification  for  the  modules  is  a  problem.  Other  Issues  concern 


HI 


utr  and  trust  and  fell*  selection  criteria  for  pilot*.  Tha 

erlutU  nay  change  tittc  the  introduction  of  U  lystawa  ■ 

2.3.  caiMt  fnliMtiMt  •  MMitl  BUwUai 

Currant  engineering  ainag— ant  and  progr—  procedures  do  not 
facilitate  tha  infualoa  of  U  U  tha  cockpit.  Sovernoent  and  tha  aircraft 
oowpaniea  cannot  anally  attract  geod  hi  expert*.  Avioaics  integrators  arc 
oftan  aatiaflad  with  1 ini tad  autonation  aolutiona.  Tha  craw  atation 
daalpura  aaad  authority  to  derim  tha  deval  rcpnam  of  cockpit  application*  by 
acting  aa  aadiatora  baton an  tha  uaar  and  tha  an  finest a.  Tha  allocation  of 
control  ia  a  critical  laaua  in  teasing  be twain  tha  honn  and  tha  alactronic 
pilot.  Ma  apraad  that  hoy*’  paper  on  larala  of  nmonany  provide*  a  food 
ayatanatic  approach  to  laemaaing  authority  or  liability  of  tha  BC. 


Thera  haa  bean  lota  of  acadaadc  raa aarch  on  hi  conducted  in  unimrsltias 
rather  than  in  application*  aaviroananha.  During  tha  60a  and  70a  hi  tool* 
war*  tranaitionad  to  orpaaiaationa  intaraatad  ia  avioaics  applications. 

Thaaa  tools  aro  now  ia  plnoa  and  aoonaaibln.  Tha  tools  aro  haglnnl  ng  to  work 
on  applications.  Thoy  art  neatly  relatively  aiapla  applications,  an  ham 
not  tackled  tha  mally  difficult  araaa  pat.  Bowamr,  wa  are  hagl  nnl  mj  to 
loam  nom  about  diagnostic*.  One  of  tha  prohlana  ia  that  wa  ham  distracted 
tha  acadanic  —ml l  j  away  fron  dav eloping  now  tools  and  now  rapcaaantatlon 
techniques.  Tha  amilabla  tools  Unit  no  to  ad  dr  ana  lug  only  tha  slnplar 
■Tar  ana  That*  nay  ba  a  lank  of  appropriate  tools  to  ornate  tha  largar,  aore 

oenplan,  aophiatioatod  ayafana  that  are  ultinataly  need ad. 

1.2.  hi  toohnaloan  -  OUmoolmd  Ioanna 

achieving  real -tine  hi  is  of  coarse  the  key  issue,  no  wars  particularly 
concerned  nine  about  knowledge  acquisition  and  on  china  looming.  Its  anally 
said  that  its  just  a  natter  of  tins  and  affort  before  tha  currant  United  hi 
ayntanc  asks  their  nark,  how  aver,  la  truth,  wa  probably  lam  tha . 
r apr annotational  techniques  to  do  mn  that,  hlso,  wa  are  not  aura  when  we 
will  haws  a  full  undarataading  about  how  to  put  all  tha  knowledge  in  place. 

Wa  will  ham  to  ham  nacbina  learning  if  wa  am  iaoapabla  of  doing  thn 
analynna  to  acquire  nil  tha  knowledge  within  thn  budgnts  amilabla  for 
ays  tana  damlopnant* 

y.TWiHflimr  ~  ffwwdfl1 

wa  diaonoaad  at  length  tha  roqu* canasta  for  pilot  nodolo.  wo  all  agreed 
that  work  an  pilot  nodnls  In  needed,  in  warn  uncertain  about  how  oppressive 
thin  work  Will  we  bo?  Will  it  bn  ns  aggr«*eive  an  thn  work  on  error 
Monitoring  and  nutountic  pilot  error  correction?  Should  wo  anticipete  tha 
pilot's  needs  nod  gim  the  pilot  hi  support  whether  its  wanted  or  not.  These 
are  research  Issues  that  need  attsntion.  Tha  oachinea  than*  alma  will  hava 
to  bo  able  to  loam  booauaa  of  the  knowledge  acquisition  problan.  ht  wa 
should  not  allow  than  to  loam  in  nid-niaaion  and  than  suddenly  do  aonathing 
new.  Learning  should  bn  dona  during  h  aisaioo,  but  act  applied  in  the  sane 
ni salon.  Learning  should  bo  part  of  thn  damlopnont  effort.  Tha  learning 
should  ba  takaa  book  to  base  to  bo  certified  for  us*  ia  the  air.  after  tha 
debrief,  tha  laamiag  will  bn  fad  into  a  bigger  *  rates  with  a  batter  picture 
of  the  world.  The  apatan  learn*  that  "See,  new  knnring  mat  I  know,  I  can 
draw  ovaa  bettor  conclusions”,  but,  wo  am  going  to  ham  to  ham  batter 


pilots  models  sad  bottsr  learning  sods  is  to  rsslis*  thsss  systems,  with  a 
larger  knowledge  base,  Ons  of  ths  political  issues  wo  identified  wu  that 
tailoring  ths  system  to  this  pilot  is  s  ssttor  of  configuration  control.  If 
sach  pitot  has  his  own  systam  that  laarns  and  works  for  him,  than  all  tha 
ijatsMS  will  ha  diffarant  with  no  Configuration  control.  Nora  importantly, 
bad  habits  cold  ba  introdnead  into  ths  systas.  Certification  is  a  big 
problaai  for  cemmarcial  aircraft*  It  say  ba  lass  so  for  combat  aircraft,  hat 
STANIVAL  in  tha  UBAF  has  a  strong  hold.  STAJftVAL  does  not  want  diffarencas 
and  saaks  to  redoes  those  difference. 

2.1.  Cockpit  tsmlloaticmm  -  Stats  of  Kmoulsdoa 

Ms  fait  that  ths  currant  state  of  AX  will  haws  an  impact  on  tha  cockpit. 
Tha  things  we  are  doing  now  will  ba  flight  tasted  in  research  environments . 
AX  is  also  influencing  avionic  systems  development  within  tha  companies.  On 
a  simple  level,  it  is  not  as  yet  clear  how  AX.  systems  will  ba  used.  But  we 
think  that  the  current  state  of  cockpit  technology  can  handle  ths. currant 
state  of  AX  systems.  However,  what  nay  be  an  important  issue  is  that  this 
first  generation  of  AX  systems  will  only  handle  uncertainty  using 
" - iwmnn 1 ~IT1 —  action".  Ha  do  not  have  the  knowledge  to  go  msfeh  further. 


2.2. 


There  ia  a  need  for  better  understanding  of  the  requirements  for  BC 
functions.  Current  projects  address  the  level  of  technology  that  we  have  in 
place  right  now.  To  go  further,  we  will  need  to  build  rapid  prototyping 
systems  that  will  enable  us  to  look  ahead.  It  can’t  be  dome  by  mission  and 
function  analysis  alone.  Ha  will  need  to  use  simulation  end  rapid 
prototyping  facilities  to  gain  experience  with  the  cockpit  implications, 
finally,  there  ia  a  whole  series  of  soft  issues  that  are  herd  to  get 
management  to  pay  for.  Trust  is  one  of  them.  Political  impact  is  another. 
Also,  the  emotional  needs  of  tha  pilot  during  combat  need  to  be  addressed. 
All  these  ere  things  that  engineers  tend  to  ignore  but  require  study. 

2.3.  Cockpit  Xltoettomm  -  fotemtiml  Ptomotlnma 

We  concluded  that  an  essential  feature  is  communication  between  SC  end 
man.  Only  by  establishing  that  cress  ml  cation  can  you  achieve  trust  between 
tha  two.  Having  established  trust,  through  communication,  the  type  of 
information  that  you  want  haa  to  communicate  awareness  of  both  ths  tactical 
and  strategic  implications  of  any  decision. 


3 


1.1.  AX  fsdmalow  -  Stmts  of  Kmowladee 

Thera  was  a  very  high  degree  of  consensus  among  ths  group.  This  was 
rathsr  surprising,  sines  ths  major  theme  that  emerged  wee  the  gulf  in 
understanding  between  cockpit  designers  and  people  within  ths  AI  community. 

Us  agreed  that  there  la  a  very  good  understanding  of  well  scale  problems 
with  demonstrable  solutions.  But  these  tend  to  be  restricted  to  LXSF  or 
LISP-type  languages  on  workstations  with  speed  limitations.  AI  is  dsfinitely 
onto  sosm thing  tangible,  but  only  on  e  small  scale. 

1.2.  AI  Technology  -  Osremotoed  lews s 

Of  the  unresolved  issues  in  AX,  ths  arss  ehae  tends  eo  ba  technically 
avoided  ia  V  «  V.  It  is  a  crippling  problem  even  on  small  scale  systems  but 
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tt  Isn't  really  iwiriil.  it  la  also  a  problaa  in  triuftttiat  froe  »ork 
lUBaM  to  Ad*  sad  *»*#>*  aaehlaea.  let  war  poop  la  hews  attaaptad  to  do 
m  t*U  that  iatmgisiat  redundancy .was  ooa  aethod  of  addressing  the 
pcoU-ap.  of  tjrast.  " Bet  gadHOdaacr  90000  0000  aota  technical  pxoblsaw. 

Finally,  wa  felt  that  tha  Hfrtnaoat  for  looming  or  tailor  lap  of  tha  systaa 
to  tha  pilot’ a  expertise  raoalna  as  unrweolwad  laoao. 
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there  will  haws  to  ha  a  blaad  of  heuristic  and  algorithmic  approache* . 
Achieving  apaad  through  aora  sophisticated  arehltaetoraa  la  aaaa  aa  a 
panaoeat  aoat  people  who  aaggaated  ehla  wara  aot  praparad  to  discuss  tha  v  a 
▼  iooooo  at  all* 


tha  ax  ecaaoaity  have  of  farad  Mall  aealo  doooaatrabla  ayetaae  tor  tha 
cockpit  doolftoro  which'  00M  to  bo  of  aooa  value.  Bat  thora  la  no  elaar 
understanding  batueae  tha  U  ooaaaialty  and  tha  cockpit  design  coaaualty  of 
aaeh  others  capabllltlao  and  roqulrtanto.  Ohara  tha  blaaa  11  aa,  wa  ara  not 
really  aora.  Thara  la  no  elaar  ooaaaaaoa  of  tha  aealo  of  problaao  to  bo 
addraoood.  a  cockpit  daalgser  oaya  "I'va  pot  thla  anoraoua  coaplex  problaa; 
you've  pot. tha  bralao  aad  tha  technology,  a how  aa  what  you  can  00".  Whereas, 
tha  AX  paraon  la  *mt  conaolooa  of  the  U at  tad  capabllltlaa  of  what  la 
available  at  tha  aaaaat* 


2.2. 


thara  lo  a  parados  that  laada  to  tha  unresolved  laauaa.  firstly,  wa 
aaad  to  receive  what  tlaa  aealo  la  to  ha  addroaaad.  xf  you  want  acaa thing 
hare  aad  now,  oa  a  aaall  aeala  aad  with  aa  advisory  capability,  than  soaa 
aort  of  aapart  ayataa  Bight  ha  able  to  dollvar  tha  poodo.  If  you  want 
something  leap  tarn  aad  all  awbraolnp,  thaa  that  ralaaa  anoraoua  questions. 
Tha  taatlnp  aw t ho do  to  ba  uaad  aad  tha  awldoaea  required  to  prorw  that  you 
hawa  dona  noaothlnp  uaaful  atUl  naad  to  ba  raaolvad. 


2.3. 


•MBtlal  odsaakhoaa 


Thara  la  a  huge  aaad  for  a  dlalopua  batwaoa  tha  uaor,  tha  daalgaar  and 
*1  coaaunlty  to  daflaa  optimum  pay-offo.  MO  ara  la  a  situation  of  hating  to 
daaoaatrata  tha  capabllltloa  of  doing  aoaathlag  which  is  going  to  require  big 
money.  Oontlaolng  tha  political  aa china  that  you  naad  that  aonoy  la  going  to 
bo  difficult  haoauaa  tha  ooekpit  daalgnarn  aad  tha  AX  paopla  ara  calking 
dlffaroat  laapaagao.  wa  naad  to  idaatify  wans  phaaod  eangibla  prograaaa 
addressing  Mat  la  agrwod  to  bo  tha  roal  problaa. 


halfway  through  wo  roaliaod  wa  waatad  to  aako  aa  asaartloa.  Tha 
aaaartloa  was  that  wa  don't  wast  aa  alaotroole  eraMaabar.  What  wa  want  la 
aa  latalllgoat  aircraft  that  aupporta  tha  aaa's  functionality.  Wa  think  that 
is  wary  important.  Wa  fait  that  tha  anthr opooentr 1c  wiaw  of  tha  electronic 
crewmember  la  aa  laapproprlata  pointer  to  tha  way  wa  should  go  ahead.  A 
saoond  ganaral  issue  la  that  until  wa  actually  understand  what  tha  rola  of 
eho  huaaa  ia  in  tha  alreraft,  wa  ara  actually  "dead  in  tht  water*.  Unless  «a 
can  daflaa  what  it  is  tha  aaa  dost  now,  wa  cannot  do  any  function  allocation 
between  aaa  aad  aa china.  Unless  wa  hawa  a  fair  description  of  ann'a 
functionality,  wa  ara  dead. 
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What's  th*  etat*  of  tha  are  in  AI  technology?  Our  thought*  war*  similar 
to  thoa*  of  eh*  Syndicat#  Ho.  3.  we  actually  Know  how  eo  do  seat  Chine*  w* 
n**d  for  aircraft  in  feh*  laboratory  environ** at.  There  are  axaeplar*.  so 
Chat’*  no  problaa.  Th*  hardware  and  software  w*  n**4  eo  deliver  thoa* 
ayaeaaw  ia  available  now. 

1.2*  hi  Tfihanlrwr  -  Obraaolved  leeuee 

Th*  unr**olv*d  iaaua  ia  haw  w*  actually  put  what  w*  know  in  hardwar*  and 
aoftwar*  technology  together.  What  w*  do  n**d  ia  ayat***  architactur*  for 
aircraft  to  b*  abl*  to  do  th***  thine*  1“  raal  tiaa.  everybody  put  up 
diagraaa  which  ahowad  that  in  th*  aiddl*  of  th*ir  ayatawa  waa  th*  Cxacutiv*. 
M*  don't  think  any  of  ua  know*  what  th*  Ixecutiv*  r sally  look*  lik*.  Chat 
doe*  th*  Executive  do  and  bow  do**  it  really  work?  That  ia  something  that  ia 
really  quit*  difficult.  It  nay  be  th*  last  thing  w*  achiev* ,  but- in  fact  a 
lot  of  things  won't  work  until  w*  can  do  that.  Lastly,  th*  verification  and 
validation  problaa  needs  to  b*  resolved. 

1*3.  hi  Twlanliwy  -  Potential  Dirwction 

So,  where  do  w*  go  with  hi  technology?  We  focused  on  th*  cognitive 
iaaua*.  On*  of  our  discussions  concerned  th*  relationship  between  th*  huaan 
and  eh*  syataa;  th*  human -aachlne  interaction  and  not  th*  huauui-wa china 
interface.  It  ia  •  eognifeiv*  syataa  that  wa  need  as  wall  a*  display 
technology  end  action  technology .  Our  second  point  concerned  parallelise. 

We  have  talked  repeatedly  about  needing  a  parallel  solution.  We  think  it  is 
actually  very  hard  to  think  about  parelleliaa.  hasuaptions  are  Bade  about 
th*  huaan  being  a  parallel  processor.  Actually,  aan  is  a  serial  thinker.  No 
eatter  what  the  processing  does,  aan  actually  thinks  in  a  serial  Banner. 
Hatheaaticiana  recently  triad  to  take  conventional  parallel  problaes  and  asks 
thaa  go  parallel.  They  ware  quite  "brain  failad"  and  found  it  vary 
difficult.  One  way  of  dealing  with  parallelise  i*  to  asaain*  the  functional 
partitioning.  Whet  i*  th*  appropriate  functional  partitioning  of  th*  problaa 
that  allow*  you  to  do  par alls lisa?  Finally,  we  decided  that  the  object  of 
building  these  systaaa  ia  to  increase  th*  survivability  of  th*  weakest  pilots 
and  not  necessarily  to  increase  the  perforaance  of  th*  best  pilots. 

Therefore,  this  technology  should  be  used  to  disseainata  expertise. 

2.1.  Cockpit  lafltotlaaa  -  State  of  Knowledge 

There  has  been  significant  advances  in  display  technology.  We  were  less 
convinced  that  there  has  been  significant  progress  in  th*  action  side  of  the 
controls.  W*  have  condensed  display*  but  w*  seaa  to  have  as  aany  switches  as 
we  used  to  have.  Also,  while  th*  technology  say  improve,  wa  aannot 
realistically  expect  our  pilots  to  be  any  better  chan  th*  pilots  of  latter 
day*  in  their  ability  to  perform.  We  already  select  th*  boat  people. 
Technology  is  going  to  lamp  ahead  in  orders  of  magnitude.  But  the  ability  of 
the  pilots  1*  not  going  to  change  vary  significantly.  Huaan  physiology  and 
psychology  are  alrsady  close  to  th*  limits.  Thar*  is  not  such  more  we  can  do 
to  iaprov*  the  pilots. 


2.2.  Cockpit  T—lloaHoaa  -  Par— olwad  laaaaa 

On*  of  the  iaauoa  ia  how  eo  design  th*  human -me chin*  interface  component 
for  future  aystaaa  with  current  problaa*.  Bow  do  we  got  people  who  use 
interfaces  for  current  weapon  aystaaa  eo  try  to  ehli.k  about  how  they  are 
going  to  control  future  weapon  systems?  We  end  up  trying  eo  design  a  futur* 
weapon*  syataa  with  today's  type*  of  lntarfaca  and  asthoda  of  control.  The 


previous  generation  u*  United  by  eh*  way  in  which  ebay  her*  been  taught  to 
aaa  their  equipment.  It  la  not  that  they  can’t  do  bettor,  but  that  they  have 
boar  taught  differently  and  can’t  aadaratand  the  technology .  toother 
interacting  «ypi  we  dlacuaaad  Concerned  the  o  one  apt  of  the  virtual 

cockpit  and  aye  polat-of-fegard  switching.  It  can  be  argued  that  the  apatial 
dlatrlbotloe  of  awltehea  la  the  cockpit  confer*  particular  benefits.  »e 
don’t  know  whather  w*  can  get  rid  of  apatial  action  control.  In  an 
enargancy,  it  nay  help  not  to  have  to  look  at  the  coat role  aelected.  The 
idea  that  whatever  you  are  doing  now  la  at  the  focua  of  your  attention  uwa 
attractive  but  la  high  action  atataa  it  nay  not  be  appropriate. 

2*3.  Cockpit  tool tontines  -  Potential  Direct  lone 

What  didn’t  cone  out  in  the  dlaeueeioaa  waa  the  problana  and  concepta 
aaaodated  with  tango  of  action.  Quite  clearly  we  need  to  diacriniaate 
between  activitiee  involving  very  high  tango  changer  of  eventa  and  eetivitiea 
that  are  very  alow,  we  need  to  nake  than*  dlacriainatioaa  if  we  are  going  to 
decign  effective  aolutlcna.  voice  control  will  not  be  ueed  for  very  high 
tanpo  eetivitiea  where  hand* -on-* tick  type  of  action*  are  needed.  Inalde  a 
racing  car  cockpit  there  are  very  few  instruments.  the  driver  uaea  voice 
chwnunl cation  at  very  low  band  width*  for  a  lot  of  hia  atrategic  control, 
such  aa  what  he  1*  going  to  do  on  the  next  lap.  doing  into  corner*,  the 
driver  apeak*  very  alowly,  with  gape,  and  weapon*  back  in  the  pita,  with 
naaalve  cccputing  facilities,  tall*  hia  what  ha  need*  to  know.  Thi*  nodal 
could  be  uaed  to  taka  scam  of  the  information  proeeaalag  from  the  pilot.  Our 
aeooad  point  ooncama  the  appropriateness  of  how  we  think  about  the  pilot  in 
the  future  cockpit.  Perhaps,  we  aheuld  be  thinking  not  ao  ouch  about  eh* 
pilot  Of  a  two- teat  aircraft  (pilot  +  SC)  but  of  the  pilot  on  the  bridge, 
like  the  Captain  on  the  bridge  of  a  a hip.  the  Captain  dlaaaninaea*  authority 
and  raeponalbility  down  through  a  eoanand  hierarchy  to  carry  out  and  complete 
task*  within  tho  constraint*  which  they  specify.  Ac  Captain's  pleturs  is 
vary  naoh  a  global  vlaw  rathsr  than  a  data ilad  view.  Ibis  nay  b*  a  useful 
alternative  way  of  of  thinking  whan  wn  design  futurn  cockpit  systems,  ws 
need  to  look  at  tha  way  in  which  other  people  interact  with  oeohiaes,  and 
psopla  with  pnopls,  sad  to  ooanldar  tho  relationship*  between  people  in  terns 
of  ooneepts  like  autocracy  and  democracy,  and  lateral  or  hierarchical 
decision  struetnrna.  Our  last  point  concerns  stress.  Sts  search  has  shown 
that  if  people  don't  have  procedures  to  follow  in  Mass  of  higi  stress  their 
performance  degrades  much  more  dramatically  than  if  they  have  got  some 
automatic  procedure  to  carry-out.  Thor*  are  times  whan  we  will  need  to 
provido  activitiee  to  do-atroon  aircrew  that  don't  necessarily  help  the 
global  view  of  the  task. 


3 


i.i.  AI  Tidilip  -  ntafce  of  Knowledge  (with  gnenoolvod  Tnrni*) 

Firstly,  wn  considered  Expert  Eys  tarns.  We  concluded  that  compared  with 
automation  there  la  very  United  operational  experience  with  expert  systeaa. 
One  reason  Is  that  Expert  Systems  are  invariably  ussr-pscod  and  not  driven 
dynamically  by  the  environment.  Me  have  virtually  no  experience  with  real¬ 
time  dynamic  lnfsreaelng  mechanisms.  Ms  felt  that  other  software  engineering 
issues  need  to  bo  sddreseed  as  well  as  V  t  v.  Me  don't  know  how  expert 
ayeteme  fell.  Nhet  are  the  failure  eodos  of  expert  systems?  How  do  we 
measure  toe  reliability  Expert  fystsms.  Mhst  about  redundancy?  Do  you  oak* 
a  triply  redundant  Expert  Systems  where  w*  reproduce  th*  software  and  then 
just  hops?  Do  you  change  th*  expert  knowledge  In  each  one?  Hew  do  you  asks 
them  fault  tolerant?  In  whet  way  are  they  fault  tolerant?  kra  there 
different  kinds  of  fault  tolerance  than  In  ordinary  systems?  On*  suspects 
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that  this  is  tha  case.  Bow  do  you  Bain  tain  thaw  la  the  fiaid  and  carry  out 
configuration  control?  He  know  that  they  giv*  you  tha  oppc.  ~t  unity  to  modify 
knowledge  basaa  and  aoftwara  much  wora  raadlly  as  situations  ohanga,  and  aa 
tha  world  changes.  Biowledge  acquisition  ia  widaly  agraad  to  ba  i  major 
problem- area.-  Ona  of  tha  raaaoaa  ia  that  thara  ara  very  faw  caal  coabat 
axparta  ia  aodam  air  war  far  a-  Ha  will  hawa  to  find  officiant  aad  coat* 
affective  methods  to  develop  arpartiaa  and  acquire  knowledge  through 
•laulatlon.  Mao,  wa  will  noad  to  aacada  othar  kinds  at  expertise  such  as 
iaaga  interpretation  aad  aircraft  systaaa  daaign  skills,  Ha  coasidarad  that 
tha  requirement  for  explanation  faoilitioa  with  dynamic  a  apart  systaaa.  In  a 
rapidly  changing  environment,  thara  will  ba  littla  tins  for  explanation  and 
ao  trust  aad  awaxaaaas  will-  ba  important  factors.  Seas  fora  of  explanation 
will  bo  important  ia  asibaddsd  training  for  building  up  confidence  off  line, 
during  dabrisfing  for  inatanca.  Ha  thought  that  rula  tracing  ia  not  an 
adequate  fora  of  axplanation.  Ha  agraad  that  achieving  tha  real-tina 
requirement  is  such  wore  than  juat  a  hardware  problaa.  Real-time  operation 
is  going  to  require  reasoning  about  tha  tine  available  to  derive  a  solution; 
anticipating  tha  aaount  of  computation  it  takas  to  arrive  at  that  solution; 
deriving  method s  which  yield  aatiafioing  solutions  early  in  tha  process  and 
converge  to  batter  solutions ,  or  that  will  always  present  tha  beat  solution, 
wa  expect  these  things  from  people  aad  need  them  from  AX  aystams. 

Next,  wa  considered  planning.  Most  of  tha  currant  work  concerns  route 
planning.  It  ia  mostly  algorithmic.  A  host  of  AI  planning  techniques  way 
become  relevant.  However,  tha  currant  technology  doesn't  yet  deal  with 
planning  whan  there  is  an  adversary  and  it  doesn't  reason  about  time. 

Finally  wa  consider ad  Interface  Technology  and  Neural  Nets.  Na  agraad 
that  tha  Pilot/Vahiela  interface  ia  critical,  that  it  probably  needs  to  ba  an 
intelligent  interface  and  that  Natural  Language  Speech  ia  a  promising 
candidate.  Neural  Nate  will  provide  acme  help  but  they  won't  aolva  all  tha 
problems.  Thara  is  a  great  uncertainty  about  what  can  ba  dona  with  Neural 
Nate.  Host  of  tha  work  in  Neural  Nats  deals  with  stationary  situations.  AI 
tends  to  ba  moat  useful  for  problems  about  which  wa  can't  specify  the  and 
point.  That  la  why  wa  have  Incremental  programming  techniques  to  develop  AI 
systems. 

1.3.  ai  T— -  Potamtial  Dirmetiomm 

HS  felt  that  tha  uncertainty  about  tha  form  and  function  of  tha  EC 
indicated  a  lack  of  clear  daaign  goal  and  unawarenass  of  what  can  really  ba 
accomplished.  These  issues  have  implications  for  tha  design  of  tha  system, 
for  tha  interaction  of  the  pilot  and  tha  system,  and  for  tha  nature  of  tha 
interface.  Should  tha  system  aid  knowledge  acquisition?  Ha  loaa  most  of  tha 
pilots  whan  thay  ara  inexperienced.  Should  tha  system  raise  tha  level  of 
knowledge  early  to  that  of  a  pilot  with  15  or  20  missions?  Clear ly ,  if  wa 
could  gut  tha  pilota  to  a  level  of  knowledge  where  tha  aurvivel  rates  ara 
much  batter,  than  that  would  achieve  a  vary  important  goal.  Should  wa  aim  to 
improve  tha  performance  of  the  expert  pilot?  Tha  system  would  have  to  ba 
designed  differently  to  achieve  that  goal.  Should  tha  electronic  crewmember 
ba  uaed  to  cover  g-induced  loss  of  consciousness?  That  implies  giving  up 
autonomy.  There  is  no  autonomy  to  ba  exercised  if  you  ara  unconscious,  what 
about  returning  tha  aircraft  whan  tha  pilot  has  bean  injured?  Ha  need  to 
focus  again  on  Improved  knowledge  acquisition  methods  for  the  real-time 
issues,  and  on  tha  aoftwara  angi near lag  issues.  Ha  agraad  that  there  are 
many  differences  between  tha  commercial  world  and  tha  military  world  as  far 
aa  tha  role  and  authority  of  tha  Expert  System  is  concerned.  On  trust  and 
confidence,  at  least  ona  of  tha  group  members  fait  that  wa  would  gat  trust 
and  confidence  the  way  wa  gat  it  now.  Ha  could  put  teat  pilots  into  the 
system,  let  them  build  up  trust  and  confidence,  and  than  transmit  that  crust 
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wd  ceafldtan  whew  to  eh*  operational  pi  lota. 

2«  2.  CmIbII  jjBUMtiqic  •  BuiMolvvi  itaati 

mm—i— aw  . 


la  the  Expert  Eyraiee  that  hare  been  developed  to  date,  aore  than  half 
of  the  Code,  aad  eoeetleee  op  to  dO«.  la  la  the  user  interface,  the 
iaterfec*  to  theae  eyataee  la  erltleal  for  the  pilot.  The  Interface  le  going 
to  be  driven  bp  what  that  eyvtee  can  do.  The  nature  of  coaenaleatloa  will  be 
driven  bp  the  role  agreed  for  the  epetaa.  Hhat  to  preeeat.  where  to  preeeat 
le.  aod  whan  to  preeeat  it  are  laaaea  that  will  alwepa  need  to  be  received. 
There  will  be  lota  of  opcloae  that  we  never  have  had  before.  He  could  cake 
it  context  depeadeae  for  laetaace.  The  honen  factor*  of  Al  apateee,  expert 
epateaw  aad  epeech  have  beaa  Inadequately  addreeeed  aad  need  attention. 


Me  were  not  too  aura  that  the  atate  of  the  art  la  all  that  high,  and  we 
were  not  aura  that  we  know  what  the  electronic  creweeher  really  la.  But  we 
know  that  planning,  dtagwocla  aad  deeieloa-eeklag  are  aa  Integral  part  of  the 
electronic  ereweeeber.  Me  aee  a  lot  of  dlf fereat  area*  la  whioh  expert 
eyataee  are  being  applied  already.  But  we  haven't  heard  a  lot  about  what  the 
Integrator  or  executive  looka  Ilka.  Mhlla  there  are  a  lot  of  knowledge  tool* 
available,  we  don't  aee  a  large  range  of  real-tiae  tool*. 

1.2.  ax  TWnhenlner  -  awaited  teases 

One  of  the  thinge  we  eee  aa  aa  aareaolved  iaeue  la  how  to  do  the 
integration  of  the  apateee  folke  with  the  hoaaa  factor*  folks.  another  lean* 
la  the  technology  lepll cat tone  of  new  devlcea.  aad  the  advance*  that  are 
occurring  la  hardware.  One  of  the  pTnhleaa  la  that  the-oeed-to-know  barrier 
often  preoludee  eooeee  to  the  lafoteatloa  you  went  aad  think  you  need, 
becauae  « one body  else  does  not  agree  that  you  need  it.  He  are  not  quite  sure 
how  to  aolva  that  problae. 

1.3.  AT  TWuhnolowv  •  Hotaatlai  Olrwotlaee 

Budgets  are  going  done.  Priori tlae  are  going  to  be  redirected,  not  only 
based  on  budget  deoieloae.  bet  oa  things  we  eaa't  predict.  Ho- body  would 
have  predicted  the  Vincennes  incident,  but  that  incident  nay  have  a  big 
baariag  oa  tha  priori ties  that  apply  to  certain  klade  of  progress*  budgets. 

He  don't  know  what  tha  asst  Incident  le  going  to  bo.  Thors  ere  going  to  be 
historical  factors  which  influence  the  direction  that  thinge  go,  eoopletely 
independently  of  our  other  ooocerne  about  budgeting.  History  is  going  to 
have  its  impact  as  wall.  Certain  probleae  are  gulag  to  drive  solution*,  a 
lot  of  attention  le  bolng  given  to  electronic  warfare  syetaaa  aad  a  number  of 
AI  applications  have  bean  established,  crow  protection,  Relaet  Mounted 
Olaplay  aad  Virtual  Display  technology  are  going  to  have  a  strong  iapact  on 
driving  whore  we  go  with  AZ  technology.  AX  le  going  to  evolve  and  bo  applied 
whether  we  guide  it  or  not.  He  felt  that  the  electronic  crown  bar  certainly 
should  address  workload  Issues  such  as  sensor  fusion,  end  issues  that  deal 
with  survivability ,  Including  both  threat  aasaaauant  aad  aultiple 
■a If unctions. 

coukuit  inasaa  *  ***** sL bssSsSb. 

One  concern  was  that  prognoele  uey  be  e  little  early.  He  really  don't 


8 


ua 


■m  clearly  all  that  la  going  to  ba  involved.  othar  factor*  ar*  going  to 
baar  on  what  happens.  Avionic*  programme  ar*  going  to  influanca  what  go** 
into  tha  cockpit.  But  tha  alactronic  crewmember  1*  certainly  going  to  hav*  a 
biginf luanca  on  tha  how  and  whan  information  gat*  di* play ad. 

2.2.  Cockpit  ^ -  CTnraaolvad  laamaa 

Tha  pilot*  associate  prograam*  will  fore*  a  certain  amount  of 
intagration  batwaan  tha  human  factor*  folk*  and  syataaa  folk*.  Tha  big 
probl«a  ia  trying  to  gat  a  definition  of  what  tha  role*  of  tha  pilot  and  EC 
really  are  in  term*  of  what  muat  the  man  do.  and  than  what  can  or  ahould  the 
electronic  crewmember  do. .  Moat  people  seem  to  agree  that  a  batter  pilot 
modal  i*  needed. 

2.3.  Cockpit  Implication*  -  Potential  Direction* 

Ha  considered  tha  question  a*  to  which  cam*  firat.  tha  chicken  or  the 
egg?  Do  w*  take  tha  system*  folks  and  gat  them  smart  on  tha  alactronic 
crewmember  or  do  we  put  the  alactronic  crewmember  export  in  with  tha  system* 
folks?  Tha  group  fait  that  it  was  probably  bast  to  put  the  electronic 
crewmember  export  in  with  the  othar  systems  folks.  He  war*  uncertain  whether 
w*  ought  to  tall  tha  pilot  everything.  He  may  know  more  than  we  want  to  tell 
him.  Certainly  we  need  to  find  tha  right  time  and  place  to  do  it.  Tha 
question  of  whether  pilots  should  ba  rmota  operators  was  discussed  rather 
thoroughly.  There  war*  a  couple  of  statements  that  wa  fait  compelled  to 
share  with  you.  On*  ia  that  if  you  make  tha  pilot  remote,  you  may  not  ba 
able  to  giva  him  tha  same  quality  and  quantity  of  information  at  the  remote 
location  that  you  could  on  board.  The  othar  problem  is  that  if  you  are  remote 
you  certainly  craata  a  vulnerability  to  jamming,  which  than  denies  that 
information  entirely. 

Finally,  we  axaminad  tha  questions  raised  at  tha  start  of  the  meeting. 
Thera  were  soma  interesting  comments: 

( 1 )  Question:  Is  tha  pilot  always  in  control? 

Answer:  No,  not  always  in  control,  but  the  pilot  is  always  in 
command.  There  is  a  need  to  separate  the  concepts  of  authority  from 
responsibility.  You  can  delegate  authority  but  you  don't  dalegate 
responsibility.  Pilots  ar*  going  to  ba  responsible  no  matter  what  you 
dalegata  to  tha  electronic  crewmember. 

(2)  Question:  What  is  the  maturity  of  the  electronic  crewmember? 

Answer:  Tha  true  associate  still  seems  a  long  way  off.  we  think 
all  this  technology  is  in  a  state  of  gestation.  He  are  eagerly  awaiting 
the  birth,  but  after  that  its  a  long  way  to  adulthood.  That  was  our  view 
of  the  maturity  of  the  child. 


(3)  Question:  What  is  the  role  of  the  executive? 

Answer:  We  felt  that  co-operation  is  going  to  be  needed,  and  that 
certainly  impacts  on  co-ordination.  But  w*  also  heard  i  lot  of  people 
saying  there  ar*  going  to  be  conflicts  that  have  to  be  arbitrated.  So, 
certainly  co-operation,  co-ordination  and  arbitration  but  w*  don't  know 
about  training.  The  configuration  control  people  hav*  said  that  there 
is  going  to  be  a  problem  with  training  and  w*  certainly  agree. 


(4)  Question;  What  is  th«  sacurity  claaranca  of  tha  alactronic 
gawbirt 

Answer)  Tha  raal  concern  was  what  do  you  do  about  a  virus.  Mow  do 
you  aasass  haaith?  How  do  wo  know  if  aoaabody  has  contaminated  the 
alactronie  craw  am  bar?  One  certainly  would  not  want  to  90  into  combat 
with  a  tick  crewmember.  Physical  control  ot  a  removable  knowledge-base 
is  probably  tha  solution  to  tha  sacurity  clearance  question. 

(5)  Question)  What  are  tha  teaming  concepts  that  are  to  be  explored? 

Answer)  There  are  lots  of  areas  in  which  humans  solve  problems  by 
delegating  certain  tasks  to  other  people  or  even  to  animals  like  tha 
pilot  dog  used  by  a  blind  person  for  navigation.  The  good  and  bad 
traits  of  other  human  and  sub-human  symbiotic  relationships  could 
provide  useful  analogs  for  cockpit  teaming  applications.  However,  the 
level  of  intelligence  will  have  a  major  bearing.  Humana  can't  or  won't 
team  with  a  worm  but  they  might  with  a  dogt 
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1.1  Al  TSCHNOLQGY  -  9TATI  OF  KNOWLEDGE 
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1.2  Al  TSCHNOLOGY  •  UNRISOLVSD  ISSUKS 


14  Al  TICHNOLOQY  -  POTIMTIAL  (MICTIONS 


2.3  COCKPIT  IMPUCATtONS  •  POTtNTUU.  DffilCTIONS 


SUMMARY  I  CONCLUSIONS 
CONCLUSIONS  —  ARTIFICAL  INTELLIGENCE 


1.  The  stit*  of  the  art  is  developed  sufficiently  Mil  to  provide  AI 
systems  for  airborne  use:  however,  they  are  mostly  devoted  to  less 
complex  and  more  easily  modeled  problems,  e.g..  utility  systems  monitoring 
and  fault  diagnosis. 

2.  The  current  complex  AI  systems  are  non-real  time.  Significant  'work  must 
be  accomplished  before  the  real  time  requirements  for  aircraft  can  be  met. 

3.  Although  some  attempts  have  been  made  at  Integrating  multiple  expert 
systems  through  the  use  of  an  executive  (e.g..  Pilot  Associate  Program), 
how  to  control  multiple  experts  is  still  not  well  known. 

4.  The  AI  tools  in  current  use  were  developed  in  the  1960s  and  '70s.  and 
there  appears  to  be  a  lack  of  new  tool  development  since  many  of  the 
researchers  are  involved  in  application  efforts. 

5.  It  will  take  a  great  deal  of  work  to  achieve  a  fully  functioning, 
operational  electronic  crewmember  and  probably  will  not  occur  until  the  next 
century. 


CONCLUSIONS  —  COCKPIT  IMPLICATIONS 

1.  A I  will  affect  pilot  workload;  effort  is  needed  to  ensure  that  AI  does 
not  increase  pilot  workload  but  that  it  leads  to  Improved  information  exchange 
and  better  workload  management. 

2.  The  cockpit  is  the  means  of  communication  between  the  pilot  and  the  EC; 
clear  information  exchange  must  occur  if  a  successful  teaming  is  to  occur. 

3.  The  pilot  must  build  up  trust  in  the  EC;  it  will  only  come  through 
increased  interaction  over  time  (l.e. .  through  training,  simulations,  and 
flight  tests.) 

4.  The  avionics  systems  will  determine  what  raw  data  is  available  for 
presentation  in  the  cockpit;  the  AI  systems  will  integrate  the  data  into 
information  packages  and  determine  how  much  and  in  what  form  the  information 
will  be  presented. 
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CONCLUSIONS  -  UNRESOLVED  ISSUES 

1.  EC's  functionality  and  the  level  of  the  EC's  autonomy  has  yet  to  be 
defined.  How  much  authority  far  indeoendent  action  will  the  oilot  be 
willing  to  assign  to  the  EC?  How  will  the  levels  of  autonomy  vary 
across  EC  functions? 

2.  The  means  for  validation  and  verification  of  AI  software  are  not  well 
known,  what  technioues  will  be  used  to  ensure  that  the  software,  which  is 
often  heuristic  in  nature,  will  behave  reliably? 

3.  The  interpretation  of  oilot  intent  is  not  well  defined  at  this  time. 

In  order  to  be  an  efficient  team,  the  EC  must  know  the  "personality" 
of  the  individual  oilot  it  is  teamed  with.  How  will  this  "oilot  model" 
data  be  obtained  and  who  will  have  access  to  it  besides  the  oilot  and 
the  EC? 

A.  The  role  of  learning  in  the  EC  may  be  the  key  unresolved  issue. 

Not  only  do  we  face  the  ouestion  of  can  the  EC  team,  but  perhaos. 
more  imoortantly.  will  the  EC  be  allowed  to  learn  in  an  ooerational 
setting?  How  will  the  newly  acaulred  information  be  integrated  into 
existing  data  bases  and  reasoning  schemes  while  meeting  the  requirements 
for  configuration  control? 

5.  The  means  of  informing  the  oilot  of  the  EC’s  decisions,  esoecially 
those  dealing  with  uncertainty,  needs  to  be  determined.  Hill  the  EC 
merely  state  to  the  oilot  that  the  selected  route,  for  examole.  has  a 
.9  orobabillty  of  success?  Will  the  oilot  be  satisfied  with  this  level 
of  exolanation  or  will  he  demand  more  information?  How  much  more? 


